AGoT: a Reasoning Framework for LLM Agents

Modern Large Language Models (LLMs) exhibit structural limitations that reduce their reliability – such as opaque reasoning processes and limited exploration of alternative solution paths – which prompt engineering techniques such as Chain of Thoughts (CoT) can address only partially. This paper introduces an Agentic Graph of Thoughts (AGoT), which extends the Graph of Thoughts (GoT) prompting technique to an agentic framework. In AGoT, LLM-powered agents leverage a dynamically-constructed graph to record their reasoning, possibly including generation, tool invocation, self-evaluation, and tool-crafting steps. Agents can explore multiple reasoning paths, possibly backtracking and extending the graph as needed, while making their reasoning process more explicit and inspectable. AGoT supports dynamic tool use and crafting: new tools can be autonomously generated at runtime when no available tool can solve the problem. We evaluate our approach on the Hendrycks MATH benchmark, comparing the proposed architecture against a Zero-Shot CoT baseline across three different Gemini models. Results suggest that AGoT can make reasoning more robust, as the multi-path exploration and tool-crafting mechanisms help agents recover from incorrect reasoning steps and address a wider variety of problems.