FlowtandBriefAI · Software · ResearchMeet Flowtand

Hugging Face Skills lets Codex run fine-tuning jobs end to end

Announcement

Hugging Face

Browse Tutorials & guides topic page →

Hugging Face published a tutorial showing how its Skills repository, which includes an AGENTS.md file, can be loaded by OpenAI Codex to run machine-learning experiments through Hugging Face Jobs. In the walkthrough, Codex inspects a dataset, proposes a training configuration and cost, submits a supervised fine-tuning job on a small Qwen model, and maintains a Markdown report with training and evaluation links.

What Hugging Face demonstrated with Codex

Hugging Face published a tutorial describing how OpenAI's Codex coding agent can use the Hugging Face Skills repository, which contains an AGENTS.md file that Codex detects automatically, to run model-training workflows. The tutorial says Skills is also compatible with Claude Code and Gemini CLI. In the example, a user asks Codex to fine-tune Qwen3-0.6B on the open-r1/codeforces-cots dataset with supervised fine-tuning, maintain a report and evaluate with the openai_humaneval benchmark. Per the post, Codex validates the dataset format, proposes hardware and parameters, estimates cost, submits the job to Hugging Face Jobs and updates a Markdown report in training_reports/ with run, Trackio and evaluation links. The post states the extension covers SFT, DPO and RL with verifiable rewards for models roughly 0.5B to 7B parameters, plus GGUF export. The quoted example HumanEval score of 0.304 belongs to the base model baseline, not to a fine-tuned result.

A realistic scenario for an ML engineer

An ML engineer who already has a Hugging Face Jobs plan and a codebase with a training script could try delegating a narrow, well-specified run to Codex: for instance, asking it to check whether an internal conversation dataset is SFT-ready, then run a short LoRA fine-tune and evaluate a base model plus the resulting checkpoint on a chosen benchmark. The tutorial's report pattern, where each run gets a title, status, run link and evaluation row, is the part most likely to save time, because it keeps the experiment record consistent. A team could also ask Codex to preprocess columns into the chosen/rejected shape the post shows for DPO compatibility. Treat this as delegation of mechanical steps, not of research judgement: the engineer should still choose the dataset, benchmark and acceptance threshold before the run starts.

How to trial it against a baseline

A team could start by reproducing a baseline the tutorial describes: evaluate the untouched base model on the chosen benchmark and record the score in the report. Then run a small fine-tune, reusing the same evaluation setup so the comparison is like-for-like. Before approving submission, read the configuration Codex prints and check hardware, estimated time, estimated cost, output repository and whether the plan covers Jobs. Afterwards, compare checkpoint scores with the baseline, but also check for failure modes: dataset validation passing while examples are malformed, evaluation leaking training data, or reports showing stale links. Keep a human reviewer signing off on each submission and on any report used for decisions. A modest success criterion is that the report accurately reflects every job and that the fine-tuned model beats the recorded baseline; if it does not, investigate data splits before accepting the result.

Trade-offs and when to hold back

Delegating runs to an agent shifts effort from scripting to reviewing, so the report quality and the approval step matter more than raw automation. Prerequisites in the tutorial include a Hugging Face account with a paid plan, a write-access token, an installed and configured Codex, and optionally the Hugging Face MCP server. Costs are GPU-time dependent, and the post's estimate for one small demo run does not generalise. The excerpt does not establish how reliable Codex's parameter choices are across datasets or whether its report stays accurate on long pipelines. A team should avoid this route for sensitive datasets, for production releases without human evaluation, or where the benchmark cannot be trusted as a proxy for real tasks; in those cases manual orchestration with explicit review gates is safer.

Context and limitations

  • Based on a short excerpt of a vendor tutorial; the full article and any linked code were not reviewed.
  • All described capabilities, costs and benchmarks come from the publisher's walkthrough, not from independent testing.
  • Hugging Face Jobs require a paid Hugging Face plan per the tutorial, and training runs consume GPU time and budget.
  • Based on bounded source text, not independent testing. Practical suggestions are editorial proposals.