This project was developed during GSoC 2024 and involved the creation of a Rocket.Chat app called LLM Prompt Editor. The app integrates Golem, a prompt editor, with Rocket.Chat's in-house AI models to provide advanced prompt editing and AI inference capabilities directly within Rocket.Chat.
- Create a Prompt Editor: Develop an LLP (Large Language Model) prompt editor using Golem.
- Decouple Inference Functionality: Replace Golem's existing OpenAPI inference with Rocket.Chat’s in-house AI models.
- Support Streaming: Implement concurrent streaming of responses across multiple devices.
- Integrate with Rocket.Chat: Bundle the Golem editor and serve it through a Rocket.Chat app.
- API Endpoints: Develop API endpoints in the app to serve the Golem build.
- Data Persistence: Store chat history in Rocket.Chat's MongoDB for future inference.
- Conversation History API: Provide an endpoint for Golem's frontend to request conversation history.
- App Commands: Create Rocket.Chat app commands to initiate workflows from the main Rocket.Chat instance.
The process of decoupling Golem's behavior from OpenAPI inference was complex due to the numerous hooks involved.
Similarly, fetching and rendering conversation data posed challenges due to the hooks.
Golem used Vite for packaging, which by default splits JavaScript into multiple files. For this project, it was crucial to minimize the number of files, requiring significant adjustments.
The demo includes a video showcasing the following features:
Screen.Recording.2024-08-23.at.10.40.05.PM.mp4
Demonstration of how the app supports concurrent streaming of responses through Golem rendered via the Rocket.Chat app.
How to start a prompt editing session using the LLM Prompt Editor in Rocket.Chat.
Visualization of how responses are streamed and handled concurrently across different devices.
Demonstration of how chat history is saved in MongoDB and how it can be retrieved for future use.
Explanation of how to use the custom Rocket.Chat app commands to interact with the LLM Prompt Editor.
Make sure you have a working Rocket.Chat server and Apps-Engine CLI for your machine. You can setup the server for your local machine from here and CLI from here.
-
Navigate to the
client
folder:cd Golem
-
Install all required packages:
pnpm install
-
Build the webpack bundle for the Excalidraw React app:
pnpm run generate
-
Execute the build.js script to generate build scripts for
bundle.ts
npm run build:golem
-
Change directory to
LLM App
and install all Rocket chat app packages :cd ../LLM App/
npm install
-
Deploy your app locally
rc-apps deploy --url http://localhost:3000 --username ${username} --password ${password}
Your username and password are your local server's user credentials .Verify the successful build by accessing the
/excalidraw
endpoint in the Whiteboard app settings. You can access the React app through the provided URL.
-
You can use the Makefile to run the server as well
make YOUR_USERNAME=${username} YOUR_PASSWORD=${password}
Make sure to replace ${username} and ${password} with the actual username and password values of your local server's user credentials. Alternatively, you can modify the Makefile directly by replacing the USERNAME and PASSWORD variables.
Additional Commands:
For build:
make build YOUR_USERNAME=${username} YOUR_PASSWORD=${password}
For deploy:
make deploy YOUR_USERNAME=${username} YOUR_PASSWORD=${password}
Follow these steps to set up your development environment using Gitpod:
-
Visit Gitpod Website:
- Go to Gitpod and click on the dashboard.
-
Login with GitHub:
- Login to Gitpod using your GitHub account credentials.
-
Create a New Workspace:
- Click on the "New Workspace" button.
- In the dropdown menu, select the repository you want to work on, specifically the
Apps.LLM.PromptEditor
repository that you've previously forked on GitHub.
-
Continue and Wait:
- Click "Continue" and give it some time to initialize your workspace.
-
Start Coding:
- After a few seconds, you'll see a fully-functional code editor in your browser.
- Feel free to start coding, making changes, and contribute to the
Apps.Whiteboard
repository.
That's it! You are now set up and ready to contribute. If you encounter any issues or have questions, refer to the Gitpod documentation or reach out to the community for assistance.
Happy coding!
Continued efforts to streamline the handling of hooks in Golem.
Further optimization of the Vite packaging process to suit the app’s needs.
Here to expresses my sincere gratitude to my mentors and provide my profile details for networking
I would like to express my sincere gratitude to my amazing mentors Gabriel Engel, Sing Li, John Crisp and Shubham Bhardwaj, for their invaluable guidance and support throughout my GSoC'24 journey 🚀. Their constant encouragement, technical expertise, and willingness to help have been instrumental in the success of this project. I feel incredibly fortunate to have had the opportunity to learn from such talented individuals. 🌟
I would also like to extend a heartfelt thank you to the entire Rocket.Chat developer community for their openness and support throughout the development process. Thank you both for your mentorship and for inspiring me to learn and grow beyond GSoC. 🙏