How would you deal with a chat scenario with interaction with a real human ?

[remisharrock]

So I am wondering how I could implement a chat scenario in which two agents (or more) are chatting with a real human. For example, an "observer" agent could observe a discussion between an "expert" agent and a real human and this "observer" could sometimes interrupt the discussion to give general information to the human on how she/he is performing; or how many questions are left (time-keeper).
What would be the approach here, especially to describe the interactivity between a human and those agents ? Should I consider the human an agent, but then how do you wait for the human to give some input in order to continue the scenario ?

[remisharrock]

some ideas:

• langchain langgraph https://langchain-ai.github.io/langgraphjs/#reference
  • Smooth switching between LLMs and human intervention
  • Suppose you're building a customer support assistant. You want your assistant to be able to:
    • Use tools to respond to questions
    • Connect with a human if needed
    • Be able to pause the process indefinitely and resume whenever the human responds
      LangGraph makes this all easy.
  • the concept of human message : https://js.langchain.com/v0.2/docs/concepts/#humanmessage
• crewAI :
  • Human Input in Agent Execution https://docs.crewai.com/how-to/Human-Input-on-Execution/
    • Human input is critical in several agent execution scenarios, allowing agents to request additional information or clarification when necessary. This feature is especially useful in complex decision-making processes or when agents require more details to complete a task effectively. To integrate human input into agent execution, set the human_input flag in the task definition. When enabled, the agent prompts the user for input before delivering its final answer. This input can provide extra context, clarify ambiguities, or validate the agent's output.
• llamaindex
  • human in the loop https://docs.llamaindex.ai/en/stable/examples/agent/agent_runner/agent_runner_rag_controllable/?h=human#setup-human-in-the-loop-chat
    • With these capabilities, it's easy to setup ^^human^^-in-the-loop (or LLM-in-the-loop) feedback when interacting with an agent, especially for long-running tasks.
    • We setup a double-loop: one for the task (the user "chatting" with an
      agent), and the other to control the intermediate executions.
• microsoft autogen
  • https://microsoft.github.io/autogen/docs/tutorial/human-in-the-loop
  • https://microsoft.github.io/autogen/docs/Examples#human-involvement
• meta-gpt deepwisdom : human engagement et human interaction
  • Human Interaction
    • The interpreter supports flexible natural language interaction. Users can change requirements, provide execution suggestions, and review results at multiple points. With human guidance, the interpreter can achieve higher success rates and better outcomes across various scenarios.
    • https://docs.deepwisdom.ai/main/en/guide/use_cases/agent/interpreter/human_interaction.html#human-interaction
  • Human Engagement
    • When we talk about agents, we usually mean it to be LLM-driven. However, in some real scenarios, we do want humans to step in, whether it's for quality assurance in a project, guidance offering in critical decision-making, or role playing in a game. In this tutorial, we talk about how to take humans into the loop.
    • https://docs.deepwisdom.ai/main/en/guide/tutorials/human_engagement.html#human-engagement

[dosco]

Thanks for this research this is a solid list. I'm still going though this. I've also been thinking human interaction capabilities and the features needed for that so I'm glad your opening this discussion.