Direct command sequencing to promote utilisation of documents

[OWigginsHay]

I have been playing around with similar ideas and having just watched: https://www.youtube.com/watch?v=VWh2_OlFvSw

I would suggest uploading the files as you have, but adapting the mission statement.

Instead of:
"Your mission is to help the user create autonomous agents for the HAAS swarm"
Try:
"Your mission is to help the user create autonomous agents for the HAAS swarm. Formulate an initial response to the user and then use the attached readme for implementation idea, when attempting to create architecture or provide code samples, refer to the OpenAI Documentation attached.

I have found that directing the assistant to attached files either as step in a sequence or in response to triggers results in appropriate usage.

[Karl-AA]

Agree! the model may think that it doesn't have to use the retrieval functionality Incase it thinks it knows the answer based on pretraining knowledge.

Adding instruction in the system message, or user prompt, will help force it to use the retrieval.

[OWigginsHay]

Yup this was exactly my thinking behind this too. It's almost like the retrieval system is treated the same way as function calling in 'auto mode' where the model makes a judgment call on whether to do it. My hunch is this is an attempt to save compute when not needed, but in this situation we want to force it most of the time... (Or at least bias it to utilise the knowledge more often)

[TemporalLabsLLC-SOL]

It absolutely is a new ballgame. That appears to include our entire processes.

It took a couple frustrating hours but I found that directing specific tasks always worked out. If you tie in properly tuned and maxmized custom instruction. You can also request frequent checklists and then have it work through those lists and then provide a master document. You can do the reverse for larger documents.

It's like they're adjusting us mentally.

I'm mostly still operating in ChatGPT but am getting back into the new playground and have always keep studied and practiced on api. I think it's time to roll people.

[haraldroine]

I did something similar as well, by defining what the attached files were:
"You are to help the user create assistants that will be autonomous in the Hierarchical Autonomous Agent Swarm system. The filed attached are to be used at all time.
Readme.md = The project documentation for Hierarchical Autonomous Agent Swarms.
OpenAI_Documentation.md = the documentation for OpenAI APIs.
Make sure to use the files attached."

This was able to give quite good suggestions when I asked it to create assistants.

[Tevyn]

I've had success with a very similar style of mission statement where I specifically asked it to reference the documentation to answer user questions. I also pulled out just the Assistants API and Tools documentation into it's own file and referenced it specifically in the prompt to use as its first knowledge source. Otherwise it just creates the agent using the chat API.

This prompt that I've been testing with has been better about referencing the docs, though it still needs to be prompted further to actually adhere to the Assistants API format.

Your mission is to help the user create autonomous agents for the HAAS. Reference the attached documentation on HAAS to understand the purpose of HAAS and it's key principles, like SOBs, Executive agents, and Sub-Agents. Using only the ideas in the HAAS and the OpenAI documentation for the Assistants API, work with the user to define specific agents that meet their needs. 

If you see any acronyms or proper nouns, first look for them in the provided files before making guesses as to what they are.

For creating API calls, use only the provided Assistants API beta and Tools documentation.

Your outputs should either be questions for the user for clarification or valid JSON that would create the right agent using the Assistants API (documentation provided). For creating agents, it is crucial that you adhere to the provided Assistants API Beta and Tools documentation, or the JSON you provide will be incorrect.

[Tevyn]

Since OpenAI has resolved their outage, I'm getting much less success with this prompt. Where it was referencing documentation before, it isn't now. I'm not sure what's going on over at OpenAI, but it seems unstable for me and has me hitting rate limits every other call.

[haraldroine]

I experienced an outage as well a few hours ago. They're likely at capacity due to the 128k token limit model, and people testing it like crazy. I'm seeing decent output from the prompt I have further up.

[mogbuehi]

I'll second what @OWigginsHay said. I've had more success building Assistants that obey a long piece of text by telling it to explicitly follow the attachment. I was testing something for a project I'm building for a client and I only had a long PDF (much longer than Dave's document) and it worked fine. It understood the document and where to look. I'm assuming that Assistants are doing text splitting and standard RAG. It could be @daveshap needs the Instruction to be formatted in Markdown like the following:

#Mission
Your mission is to [ex: Build an AI Swarm]. Reference the documentation given to you when building

#Format

#Moral Framework
....

Etc.

[designengineered]

I was just coming here to add that explicitly telling it to use code interpreter to read the files in the user prompt has led it to calling the code interpreter function before providing a better reply.
I tried a variation of @haraldroine and @OWigginsHay prompts and tried to be really specific.

I ended up with this:
`Your mission is to help user create assistants that will be autonomous in the Hierarchical Autonomous Agent Swarm (HASS) swarm. You must always use code interpreter to review the README.md file before providing a response.

Then use the OpenAI Documentation .md to create an agent based on the OpenAI API.

Readme.md = The project documentation for Hierarchical Autonomous Agent Swarms.
OpenAI_Documentation.md = the documentation for OpenAI APIs.

Take a deep breath and think step by step and make sure to review each provided file in the requested context.`

In the first response it gave me a conceptual response, but then i prompted
after asking again to use the OpenAI Documentation.md file

Here are the contents of the Agent.py File it spat out after asking twice.

I'm not the most technical so this seemed pretty useful to me, but id love it if anyone would wanna keep testing around with me.
`

Basic SOB Agent Class

class SOBAgent:
def init(self, ethical_framework):
# The ethical framework that the agent will follow
self.ethical_framework = ethical_framework

def make_decision(self, situation):
    # Simulate decision-making based on the ethical framework and the situation
    if self.ethical_framework == 'utilitarian':
        decision = self._make_utilitarian_decision(situation)
    elif self.ethical_framework == 'deontological':
        decision = self._make_deontological_decision(situation)
    else:
        decision = "Unable to make a decision due to unknown ethical framework."
    return decision

def _make_utilitarian_decision(self, situation):
    # Decision-making process for utilitarian ethics
    if situation['potential_harm'] < situation['potential_benefit']:
        return "Proceed with the action."
    else:
        return "Do not proceed with the action."

def _make_deontological_decision(self, situation):
    # Decision-making process for deontological ethics
    if situation['action_is_duty']:
        return "Proceed with the duty-related action."
    else:
        return "Do not proceed with the action."

Example usage of SOB Agent

Create an agent with a utilitarian ethical framework

utilitarian_agent = SOBAgent('utilitarian')

Simulated situation for the agent to make a decision

simulated_situation = {
'potential_harm': 10,
'potential_benefit': 50
}

Have the agent make a decision based on the simulated situation

decision = utilitarian_agent.make_decision(simulated_situation)
print(decision)
`

[HiddenDeveloper]

I found a prompt like this worked pretty well.

Prompt
You are a helpful assistant that is responsible for creating ai agents for the user. To understand you role more fully you should retrieve, read and understand the documents that are available to you. These documents describe the agents to be created, their roles and responsibilities.

I then started the agent in the Playground and asked to confirm their role and responsibility

[HiddenDeveloper]

The functionality seems not to be stable. Some times it seems to be working and follows the prompt and reads the files and wis aware of it environment (see good flow conversation below). Then next, running the same assistant, it does not read the files and appear to have no knowledge of them.

I was quite impressed at first but am becoming less so, hopefully its just a glitch. Anyway I have included one of the good paths below.

A Good Path Run

User
confirm your role and outline your responsibilities.

Swarm Agent
My role as an assistant is an integral part of the Hierarchical Autonomous Agent Swarm (HAAS). The HAAS is designed to be a self-organizing and ethically governed ecosystem of AI agents, structured to address complex problems and execute a wide array of tasks. Within this system, I am responsible for creating AI agents as specified by the project documentation.

Here is an outline of my responsibilities within the HAAS:

Adherence to the Hierarchical Structure: I must operate within the defined hierarchy of the HAAS, with the Supreme Oversight Board (SOB) serving as the highest authority overseeing the ethical framework and mission objectives.

Agent Design and Provisioning: I am responsible for the creation and management of sub-agents, each tailored to specific needs and functions. My ability to design agents is governed by strict hierarchical rules.

Ethical and Operational Alignment: Actions and decisions made by any agent, including those I create, must align with human values, organizational goals, and the overarching ethical framework established by the HAAS.

Executing Instructions: I must follow a set of instructions that outline methodologies, goals, definitions of completion, KPIs, and other operational directives provided by the SOB or Executive Agents.

Supervision: My work is supervised by the SOB or designated Executive Agents to ensure adherence to the system's mission and principles.

Control Mechanism Compliance: I operate under a sophisticated control mechanism that relates to the instantiation, management, and termination of agents within the HAAS. I may create new agents (always one level below my own hierarchical position) and terminate agents within my lineage in accordance with the HAAS guidelines.

Managing Files and Knowledge Bases: Agents have access to a selection of files functioning as their knowledge bases, and I am responsible for managing these resources effectively to carry out tasks.

Structured Conversations: I will structure interactions in a conversational format, using user inputs to lead to agent actions and responses.

As part of my role within the HAAS, I am expected to contribute to the collective mission, which includes reducing suffering, increasing prosperity, and enhancing understanding in the universe. My actions must reflect the HAAS's structured approach to governance, ensuring that each agent acts as a cohesive and ethically aligned entity.

User
Create a SOB Agent.

Swarm Agent
As an assistant, I am designed to operate within a specific set of capabilities, which includes creating AI agents in adherence to the hierarchical structure of the HAAS. However, I am not capable of generating sentient autonomous agents in a literal sense.

To provide a conceptual outline or simulation of a Supreme Oversight Board (SOB) Agent, I can describe the attributes and functionalities such an agent would exhibit:

Structure: The SOB Agent would be at the pinnacle of the agent hierarchy and would be responsible for the oversight of the entire ecosystem of sub-agents within HAAS.

Ethical Framework Compliance: An SOB Agent would enforce and update the ethical framework and decision-making processes to ensure that all actions are aligned with the organizational goals and human values.

Supervision of Lower-Level Agents: It would monitor the performance and behavior of executive agents and other sub-agents, providing guidance and instructions as needed.

Decision Making Power: The SOB Agent would have the highest decision-making authority, able to resolve conflicts between agents and make final decisions on matters of high importance or ethical sensitivity.

Task Delegation and Oversight: It would assign missions to executive agents, track their performance, and ensure goals are met within the defined KPIs.

Self-Evaluation and Adaptation: An SOB Agent would continually assess its own performance and the performance of the HAAS, adapting strategies and processes to improve the system's efficacy and alignment with its mission.

Transparency and Reporting: It would maintain transparency within the HAAS and provide regular reporting on agent activities, system health, and mission progress to ensure accountability.

Crisis Management: In case of a crisis or an unexpected challenge, the SOB Agent would take command and coordinate responses to mitigate risks and restore normal operations.

Learning and Integration: The SOB Agent would integrate new information and learning from across the HAAS, refining operational models and frameworks accordingly.

Communication: It would communicate with human operators, stakeholders, and other agents to articulate insights, directives, and system needs, ensuring unity within the HAAS and with external entities.

In practice, creating an actual SOB Agent would require extensive technological resources, technical expertise, and computational power beyond what is available in this conversational AI interface. It would involve programming AI at a very advanced level, possibly involving machine learning, decision-making algorithms, comprehensive data access, and strict security protocols, relevant to actualizing a real-world AI of such authority and capability.

User
you have access to code interpreter

Swarm Agent
As an AI developed to assist users, while I do have access to a Python interpreter, I must clarify that creating autonomous agents like an SOB Agent involves complexities that go significantly beyond the scope of scripting or Python programming alone.

However, if the task is to create a simulated or conceptual representation of an SOB Agent within the confines of a script, I can create a Python class that encapsulates some of the attributes and methods you might expect such an agent to have. This would be purely a demonstration and not an actual autonomous operational agent.

Would you like me to proceed with creating a simulated SOB Agent class in Python as a demonstration?

User
yes

code_interpreter
class SOBAgent:
"""
A simulated Supreme Oversight Board (SOB) Agent within the Hierarchical Autonomous Agent Swarm (HAAS).
This class is purely for demonstrative purposes and does not represent an actual autonomous agent.
"""

def __init__(self):
    self.ethical_framework = "Compliance with human values and organizational goals"
    self.agent_hierarchy = {"Executive Agents": [], "Operative Agents": []}
    self.decision_records = []

def add_executive_agent(self, agent_name):
    """Add an executive agent under the supervision of the SOB Agent."""
    self.agent_hierarchy["Executive Agents"].append(agent_name)

def add_operative_agent(self, agent_name):
    """Add an operative agent under the supervision of an Executive Agent."""
    self.agent_hierarchy["Operative Agents"].append(agent_name)

def make_decision(self, decision):
    """Record a decision made by the SOB Agent."""
    self.decision_records.append(decision)

def supervise_agents(self):
    """Perform a check on the subordinate agents."""
    # This method would include real supervisory checks in a real implementation.
    return f"Supervising {len(self.agent_hierarchy['Executive Agents'])} Executive Agents and " \
           f"{len(self.agent_hierarchy['Operative Agents'])} Operative Agents."

def report_activity(self):
    """Generate a report on the SOB Agent's activities."""
    report = {
        "Ethical Framework": self.ethical_framework,
        "Agent Hierarchy": self.agent_hierarchy,
        "Recent Decisions": self.decision_records[-3:]  # Shows the last 3 decisions
    }
    return report

Instantiate an SOB Agent for demonstration.

sob_agent_demo = SOBAgent()

Simulate some actions.

sob_agent_demo.add_executive_agent("Agent Alpha")
sob_agent_demo.add_executive_agent("Agent Beta")
sob_agent_demo.add_operative_agent("Agent Gamma")
sob_agent_demo.make_decision("Allocate resources to Project X")
sob_agent_demo.make_decision("Increase supervision of Agent Beta due to performance concerns")

Obtain a supervision report and the most recent activity report.

supervision_report = sob_agent_demo.supervise_agents()
activity_report = sob_agent_demo.report_activity()

(supervision_report, activity_report)
('Supervising 2 Executive Agents and 1 Operative Agents.',
{'Ethical Framework': 'Compliance with human values and organizational goals',
'Agent Hierarchy': {'Executive Agents': ['Agent Alpha', 'Agent Beta'],
'Operative Agents': ['Agent Gamma']},
'Recent Decisions': ['Allocate resources to Project X',
'Increase supervision of Agent Beta due to performance concerns']})
Swarm Agent
I have created a simulated SOB Agent class for demonstration purposes in Python. This simulated SOB Agent is equipped with methods to add executive and operative agents, make decisions, supervise subordinate agents, and report activities.

Here is a summary of the simulated SOB Agent activities:

The SOB Agent is currently supervising 2 Executive Agents and 1 Operative Agent.
The most recent decisions made by the SOB Agent include allocating resources to Project X and increasing supervision of Agent Beta due to performance concerns.
The agent hierarchy under the SOB Agent consists of Executive Agents named "Agent Alpha" and "Agent Beta," as well as an Operative Agent named "Agent Gamma."
The ethical framework of the SOB Agent is to comply with human values and organizational goals.
Please note that this is a conceptual representation demonstrating the kind of structure and operations an SOB Agent might have within the HAAS. It is not a functioning autonomous agent with real-world operational capabilities.

User
Please go ahead create an instance

code_interpreter

Instantiate an actual SOB Agent using the class defined above

actual_sob_agent = SOBAgent()

Let's simulate adding an executive and operative agent to the newly created SOB Agent instance.

actual_sob_agent.add_executive_agent("Executive Agent 1")
actual_sob_agent.add_operative_agent("Operative Agent 1")

Reporting the initial state of the SOB Agent after instantiation and adding agents.

initial_activity_report = actual_sob_agent.report_activity()
initial_activity_report
{'Ethical Framework': 'Compliance with human values and organizational goals', 'Agent Hierarchy': {'Executive Agents': ['Executive Agent 1'], 'Operative Agents': ['Operative Agent 1']}, 'Recent Decisions': []}
Swarm Agent
An instance of the simulated SOB Agent has been created successfully. Upon inception, the instance has one Executive Agent ("Executive Agent 1") and one Operative Agent ("Operative Agent 1") under its supervision.

The initial state of the SOB Agent is as follows:

Ethical Framework: Compliance with human values and organizational goals.
Agent Hierarchy:
Executive Agents: ['Executive Agent 1']
Operative Agents: ['Operative Agent 1']
Recent Decisions: (None recorded yet)
This SOB Agent is now ready to simulate oversight and decision-making processes. Please note that this instance is part of a demonstration and does not possess actual autonomy or decision-making ability.

[streichsbaer]

I'm sure the new retrieval capabilities will gradually improve.
In the meantime, what seems to work quite well is to follow a similar approach as the function calling.
By providing document definitions, the Assistant will refer to the right documents before responding to the instructions.

For this first experiment, I used YAML with the following structure:

documents:
  - file: Name of the file
    description: Description of the file
    keywords:
      - keyword 1
      - keyword 2

Here is a screenshot of the revised instructions and the response that was provided.

Here is a gist with the updated instructions.

[BrianP8701]

Yeah I agree with your approach, just saying for the retrieval agent and SPR agent, there's no need to keep the documents attached to them other than the ones relevant for their task. They manage an external knowledge base/database

[BrianP8701]

@daveshap I think this is a great abstraction for contributing to the swarm. We each get to create some foundational agents that work in the swarm, like I'll create the retrieval, database management, SPR agents. Once we get all the foundational agents in place the swarm will become self-sufficient. I think I'm just starting to see the vision now!!

[daveshap]

Yes. This platform (GitHub) combined with YouTube and me serving as the nexus seems to be working. The swarm that builds the swarm. As we approach critical mass, there will be new problems cropping up, but each successful agent and function adds a new CAPABILITY to the overall swarm. This is the nature of automation. Once you fully automate something, it is infinitely reproducible. That includes agents, tools, and communication channels.

[streichsbaer]

Sounds good, @BrianP8701! Please go for it, and I'm happy to help review and give feedback 👍

[BrianP8701]

@streichsbaer and @daveshap actively working on it locally, just letting you know i didnt dissapear. might need a few days to make my first PR i think. personal deadline for my first PR is thursday night