Agents and DAG

[beddows]

Been thinking a lot about DAGs lately and how they could contribute to agent processing and observability. Came across two recent papers:

• VillagerAgent: A Graph-Based Multi-Agent Framework for Coordinating Complex Task Dependencies in Minecraft
• On the Diagram of Thought

VillagerAgent uses a DAG for coordinated action, while DoT employs it for structured thinking. I don't see why both approaches couldn't be combined - a DAG within a DAG. Probably slow and expensive, but more performant for complex tasks.

I get the impression that Ax is closer to DoT. When using Ax agents calling agents, their actons could dynamically (but sequentially, via CoT) be forming a DAG (I'm not sure if they would consistently be acyclic, or if they even need to be). I'm assuming maxSteps in AxGenerate limits how large the chain of thoughts can get (to keep it bounded).

I'm just wondering (out loud) if there's potential for parallel processing and if DAG could assist with that: "Directed Acyclic Prompts" (DAP). A structured set of steps and key values stored in memory, rather than unstructured chat history content (which I suspect is taking up an unreasonable amount of context).

It would be great to have a simple graph visualization of actions taken by agents, to better understand the task breakdown. Something like d3-dag or even mermaid. Just a (chain of..) thought, would welcome other views.

[dosco]

There a lot here also thanks for the two papers will add to my readings. In short yes Ax agents calling agents is a sort of a DAG since the agents are calling agents which can call more agents and these agent calls can be parallel of serial. At a high level we're doing a splitting of the content to help focus the LLMs which are trained on shorter tasks using something like o1 which is trained on long horizon tasks will allow more complex long tasks (aka richer graphs). Yes a visualization would be helpful. Have you looked at Ax Crew a project that allows you to define dag workflows using ax agents using yaml and more.

https://github.com/amitdeshmukh/ax-crew