feat: Separate Knowledge system + Multi-Agent RAG Optimization

[azep-ninja]

Relates to:

Risks

Low Risk, as the new RAG system is completely separate, and must be enabled by the user to use as of now.

Background

What does this PR do?

What kind of change is this?

Improvements (misc. changes to existing features)

This improves the knowledge system by separating it from the memories table, while allowing users to better manager their knowledge and have true separation of actions.

Additionally, allowing multiple agents to "share" knowledge to avoid duplicating entries for better optimizations.

All database adapters were updated to have the new functions to create/get/search/delete knowledge.

Why are we doing this? Any context or related work?

This change is a critical change as it helps the agent scale, reduce database overlap, and separates the agents actions, to ensure proper segregation.

Additionally, this also allows users to create a knowledge folder of txt/md files and reference them in their character knowledge, without having to paste the full contents in character file.

Documentation changes needed?

My changes require a change to the project documentation.

Minimal changes to exiting knowledge docs, to outline how to enable the new rag knowledge system.

Testing

Where should a reviewer start?

Detailed testing steps

• Enable the new rag knowledge
  

• Create a "knowledge" folder within your characters folder
  

• Add files in the root of knowledge, or create additional subfolders for organization

• Set your knowledge items in your character file directly, or objects specifying the path, and if it should be a shared item or not, depending on multi characters or not.
  

• Start agent(s) as normal, and watch knowledge get processed!

Screenshots

Before

After

Knowledge table with information and split into chunks, along with main document, shared documents, or agent specific documents in the database.

Deploy Notes

Database changes

• New knowledge table is created

Deployment instructions

Discord username

ninja_dev

[LinuxIsCool]

Trying to run this locally. Getting:

 ["⛔ Error parsing character from /characters/character.character.json: Error: Character configuration validation failed:\nknowledge.0: Expected string, received object\nknowledge.1: Expected string, received object"]

[azep-ninja]

Error parsing character from

@LinuxIsCool , good catch. Updated the character validation now. Can you pull the latest version and try.

Updated "package/core/src/environment.ts" file within CharacterSchema, replacing "knowledge:"

With

knowledge: z.array( z.union([ z.string(), z.object({ path: z.string(), shared: z.boolean().optional() }) ]) ).optional(),

[LinuxIsCool]

@azep-ninja

I'm running it now without errors on the latest version, however, it's not working as expected. The agent is running but there is no prompt in the console.

I am seeing:

["✓ REST API bound to 0.0.0.0:3003. If running locally, access it at http://localhost:3003."]

However, I'm not sure how to access the REST API.

And when I try the client with pnpm start:client

Then, I see my agent in the dashboard, but there is not response when I post a message, and I see in the client console:

> eliza@ start:client /home/ygg/Workspace/Eliza/akemi_rag
> pnpm --dir client dev


> [email protected] dev /home/ygg/Workspace/Eliza/akemi_rag/client
> vite


  VITE v5.4.11  ready in 158 ms

  ➜  Local:   http://localhost:5173/
  ➜  Network: use --host to expose
  ➜  press h + enter to show help
(node:118597) ExperimentalWarning: CommonJS module /home/ygg/Workspace/Eliza/akemi_rag/node_modules/tailwindcss/lib/lib/load-config.js is loading ES Module /home/ygg/Workspace/Eliza/akemi_rag/client/tailwind.config.js using require().
Support for loading ES Module in require() is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
6:35:05 PM [vite] http proxy error: /agents
AggregateError [ECONNREFUSED]:
    at internalConnectMultiple (node:net:1128:18)
    at afterConnectMultiple (node:net:1693:7)
6:35:06 PM [vite] http proxy error: /agents
AggregateError [ECONNREFUSED]:
    at internalConnectMultiple (node:net:1128:18)
    at afterConnectMultiple (node:net:1693:7) (x2)
6:35:08 PM [vite] http proxy error: /agents
AggregateError [ECONNREFUSED]:
    at internalConnectMultiple (node:net:1128:18)
    at afterConnectMultiple (node:net:1693:7) (x3)
    ```

[LinuxIsCool]

The client is now working properly after nvm use latest and re-install and re-build. So I will now test out the rag functionality.

[LinuxIsCool]

The RAG seems to be working!

Although now I have added a 5MB text file and the RAG system is currently hanging after about 5 minutes. I'm hoping it finishes processing soon and is not hanging indefinitely.

[azep-ninja]

The RAG seems to be working!

Although now I have added a 5MB text file and the RAG system is currently hanging after about 5 minutes. I'm hoping it finishes processing soon and is not hanging indefinitely.

Hmm, a 5MB file isn't that huge.

Let me know if it ever succeeds or doesn't and I can replicate to see if it's a bug.

In my tests I didn't have any humongous files so could be some need for optimizations of course.

But glad to hear it did work!

[LinuxIsCool]

I think testing with big files is a good idea. It does seem to hang. But when I restart the agent then the knowledge is loaded. Are there ways that it could be sped up?