The inspiration behind Orator.ai is deeply rooted in our ability to learn and our fascination with self-awareness and self-reflection. At one point or another, we've all felt the need to learn a skill but never knew where to start. The ability of public speaking isn't just a skill we build for ourselves but is something that can motivate and influence those around us. However, it may be hard to find what's right for us in self-learning and growing through self-reflection. 🤔
We embarked on a journey to create Orator.ai with a vision of finding that perfect fit, by harnessing technology and innovation to increase our abilities to learn and remove barriers to learning. 📷
Orator.ai isn't just an educational public speaking app, it's a way to find information for you. Orator.ai brings you from start to finish in your learning journey, providing feedback and analytics as you upload videos to improve all aspects of your public speaking skills.
- Get Started: Begin on the Landing Page and sign up for a free account, with user information encrypted by JWT.
- Dashboard: View and interact with the dashboard or press upload to provide a new video for analysis.
- Video Processing: Once uploaded, the video is compressed framewise and bitwise by ffmpeg and transcribed using Google Cloud's Speech-to-Text.
- Analysis and Feedback: API calls to VertexAI Vision and Gemini, in conjunction with LangChain for concurrency optimization and MongoDB for data storage, obtain key points, feedback, and personalized course outlines.
- Educational Content: This data is fed into the YouTube API to find the most suitable educational video.
- User Interaction: Users can view their results, past recordings, and edit previously created lesson plans to optimize their studying curriculum on their dashboard.
Orator.ai reduces the time it takes to find something relevant to each person and eliminates the barrier of entry in learning new skills efficiently.
We built Orator.ai with a variety of amazing technologies:
- Next.js: The building blocks for the front end 🧱
- Vercel: Hosting and tracking deployments for the site 💻
- TailwindCSS, SASS, Framer Motion, Three.js: Creating a stylish, animated, and user-friendly UI/UX ✨
- GoDaddy: For the amazing domain name🤩
- Flask.py: Hosting backend server and API calls
- Modal.com: Used for serverless backend deployment
- LangChain: Employed for concurrency and optimization of API calls 💨
- MongoDB: Storing user data, prompted feedback, and course data 📊
- JWT: For encrypting users' data 🔒
- VertexAI: Leveraging VertexAI Vision to analyze communication gestures 👋
- Gemini API: Prompting for feedback and generated course content 🤖
- Google Cloud Speech-to-Text: Transcribing Speech-to-Text (STT) 🎤
- YouTube Search API: Querying suitable educational videos based on generated content 🎥
Standing together as a cohesive team with a wide array of different experiences, we were delighted to bring together an app that made the most of our skill sets. Using Google's Developer Tools, we linked together processes we initially thought infeasible and completed our application in under 24 hours. The ability of different tools to function together to reduce inefficiencies and increase scalability allowed us to maintain a more diverse and flexible system with exchangeable and reusable modules.
Orator.ai isn't done yet. We're just beginning. With a future-focused mindset, we aim to bring more accurate and informative data to our users by implementing Retrieval-Augmented-Generation (RAG) and building out features for Data Analysis.
Speaking is inherently a social activity, and you're not alone. We want you to connect with a community through features, messages, or shared experiences, so you don't feel alone in your journey. Orator.ai is more than just a learning platform; our goal is to become a platform for shared knowledge and experiences.
Orator.ai is boundless. With different engineered prompts and slight changes in the codebase, users can find educational content for various purposes, especially those with Computer Vision applications (e.g., sports). The future of Orator.ai is about pushing boundaries and providing users with a new way to interact with their physical and auditory learning.
This project was created during a hackathon and reflects our passion for innovation and learning.