Skip to content

feat(backend): add SSE streaming endpoint for real-time chat responses#51

Open
SandeepChauhan00 wants to merge 1 commit intoINCF:mainfrom
SandeepChauhan00:feat/streaming-response-50
Open

feat(backend): add SSE streaming endpoint for real-time chat responses#51
SandeepChauhan00 wants to merge 1 commit intoINCF:mainfrom
SandeepChauhan00:feat/streaming-response-50

Conversation

@SandeepChauhan00
Copy link

🔗 Related Issue

Fixes #50

📋 Summary

Adds Server-Sent Events (SSE) streaming endpoint for real-time chat responses, providing a ChatGPT-like experience where users see responses appear word-by-word.

🐛 Problem

Currently, users wait 5-10 seconds staring at a loading spinner before seeing any response. This feels slow and unresponsive.

✅ Solution

New /api/chat/stream endpoint that:

  • Streams tokens in real-time using SSE
  • Shows responses word-by-word (3 words per chunk)
  • Includes start, token, done, and error message types
  • Maintains backward compatibility with existing /api/chat endpoint

🔄 Before & After

Before After
Wait 5-10 seconds Response appears in ~1 second
Full text at once Words appear as generated
Feels slow Feels instant (like ChatGPT)

🧪 Testing Performed

Tested with: curl "http://localhost:8000/api/chat/stream?query=EEG%20datasets"

Output shows streaming tokens appearing in real-time with start, token, and done messages.

✅ Checklist

  • Backend endpoint works
  • Streaming tokens appear in real-time
  • Error handling included
  • Backward compatible with existing endpoint
  • Tested locally

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Implement Streaming Response Support for Real-Time LLM Output

1 participant