FlowtandBriefAI · Software · ResearchMeet Flowtand

Hugging Face details Transformers Agents 2.0 with new iterating agent types

Announcement

Hugging Face

Browse Agents topic page →

Hugging Face published a post introducing Transformers Agents 2.0, adding two agent types that iterate on past observations alongside the existing agent, with code described as clear and modular and shared final prompts and tools. The post attributes a GAIA Leaderboard result in which a Llama-3-70B-Instruct agent outperformed GPT-4 based agents, and separates agent logic from the LLM engine. The library was later moved to a stand-alone library, smolagents, per the post.

What the Transformers Agents 2.0 post introduces

Hugging Face's post announces Transformers Agents 2.0, adding two agent types that can iterate using past observations for complex tasks, alongside the existing agent type. The framework separates the agent from the LLM engine, so an agent type can be built on any underlying model. Agents are described as programs driven by an LLM with access to a toolbox of tools, each carrying a name, description, inputs and output type used to generate the tool's usage instructions in the prompt. The post names CodeAgent (single blob of code, no iteration), ReactCodeAgent and ReactJsonAgent, which cycle Thought, Action and Observation. It also describes sharing options for community agents. Hugging Face reports that a Llama-3-70B-Instruct agent outperformed GPT-4 based agents on the GAIA Leaderboard; that is the publisher's benchmark claim, not an independent result. The post notes transformers.agents was later upgraded to the stand-alone smolagents library.

Application scenario: documentation RAG with dynamic filters

A developer-experience engineer maintaining internal documentation search could try wrapping a retriever as a Tool whose inputs include a query and a source filter, then give it to a ReAct-style agent. If documentation is split across repositories, the agent could choose which source subsets to search based on the question rather than always querying everything. A team could try this because the post describes a similar setup: a retriever tool built over a vector store of Hugging Face docs, where the agent selects sources. Whether the agent reliably picks sensible filters would need to be measured rather than assumed.

How to trial it and measure against a baseline

A team could try comparing a plain retrieval pipeline against the same retriever wrapped as an agent tool, holding the embedding model and chunk settings fixed. Use a labeled set of real questions with known correct source pages; track answer correctness, whether the correct source was retrieved, number of tool calls per question, latency and token cost. Watch for failure modes the post hints at, such as an empty filtered search returning a message asking to retry without the filter, which could silently change behavior. Add human review of sampled traces, and keep a fallback to unfiltered retrieval if the agent's filter choices degrade results.

Trade-offs, prerequisites and unknowns

Iterating agents add tool calls and reasoning steps, which likely increases latency and cost versus a single-shot CodeAgent; the post frames the latter as unable to iterate on prior observations. Using the framework requires an LLM engine callable that accepts messages and stop sequences, plus defined tools with clear descriptions. The excerpt does not detail configuration, hosting requirements or full benchmark methodology, and the publisher's GAIA comparison should not be treated as an independent evaluation. If latency budgets are tight or tasks are single-step, the simpler non-iterating agent may be preferable. Because the API moved to smolagents, teams should confirm current interfaces before adopting examples from the post.

Context and limitations

  • Based on a short excerpt of the Hugging Face post; the full article, code and benchmark methodology were not analyzed.
  • The GAIA outperformance claim is attributed to Hugging Face and was not independently verified.
  • The post notes transformers.agents was upgraded to a stand-alone library, smolagents, so API details may differ from the post's examples.
  • Based on bounded source text, not independent testing. Practical suggestions are editorial proposals.