Hugging Face details transformers upgrades for running and fine-tuning OpenAI's gpt-oss
Hugging Face
- Developer tools
- Open source
- #transformers
- #gpt-oss
- #mxfp4
- #quantization
Browse Developer tools topic page →
A Hugging Face blog post describes library changes made to support OpenAI's GPT-OSS models, including downloadable pre-built kernels, native MXFP4 quantization, tensor and expert parallelism, and batching improvements. The authors say most features should work across other major models in transformers, and that framework authors can use the code as a reference.
What Hugging Face changed in transformers
Hugging Face published a post describing upgrades made to transformers so OpenAI's GPT-OSS models can be loaded, run and fine-tuned. The authors list downloadable pre-built kernels fetched from the Hub on first use, which avoids compiling CUDA code locally; GPT-OSS uses Liger RMSNorm and MegaBlocks MoE kernels, enabled by passing use_kernels=True. Native MXFP4 support is also described: a 4-bit format with shared block scales, backed by Triton kernels, with requirements including accelerate, kernels, triton>=3.4 and NVIDIA compute capability 7.5 or higher. If unmet, the library falls back to bfloat16 at roughly four times the memory. The post also covers Flash Attention 3 with attention sinks, tensor parallelism via a tp_plan="auto" argument, expert parallelism, dynamic sliding-window layers and continuous batching. The authors say most features should work across other major models.
A scenario for an inference platform engineer
An engineer maintaining an internal inference service for a gpt-oss model could treat these changes as a menu of optional experiments rather than a turnkey recipe. A plausible first trial is the MXFP4 path, since the post claims GPT-OSS 120B fits in roughly 80 GB of VRAM when MXFP4 is active. That could let a team test a single-GPU deployment where bfloat16 would not fit, but only for hardware meeting the stated compute-capability requirement and with accelerate, kernels and Triton installed. Separately, a team whose GPUs support the custom kernels could try use_kernels=True and measure throughput at their own batch sizes, since the authors note their kernels performed best at larger batches in their test system.
How to trial it against a baseline
A team could set two runs: a stock bfloat16 baseline and one change at a time, such as MXFP4 or use_kernels=True. Record peak VRAM, tokens per second at the batch sizes actually served, and output quality judged by humans on a fixed prompt set, because these are precision and kernel changes. If moving from MXFP4 to custom kernels, confirm the documented incompatibility by checking that performance and outputs change as expected. Treat load-time log lines about which Hub kernels were selected as verification that the intended path ran, and confirm the Triton MXFP4 kernels appear in the local cache. Stop the trial if quality drops beyond an agreed threshold, and keep human review of generated output before any rollout.
Trade-offs, prerequisites and unknowns
These features are conditional. MXFP4 requires specific software and NVIDIA compute capability 7.5 or higher; without it the fallback needs about four times the memory, according to the post. Custom kernels can be opted into, but the authors state they are incompatible with MXFP4, so that combination runs in bfloat16. Flash Attention 3 with sinks is described as Hopper-only. Tensor parallelism is communication-heavy and the authors say it suits a single machine with fast intra-node links. All benchmark claims come from the authors' test system, so figures should not be assumed to transfer. The post also notes most features should work across major models, which is a stated expectation rather than a tested guarantee. Teams needing verified behaviour on non-NVIDIA hardware or older GPUs should wait for independent results.
Context and limitations
- Based on a short source excerpt; the full article was not analyzed.
- Benchmark figures come from the authors' own test system, not independent evaluation.
- The excerpt ends mid-snippet, so the parallelism material is only partially visible.
- Based on bounded source text, not independent testing. Practical suggestions are editorial proposals.