Properly Handle the Case when SSE Data is Split into Multiple Chunks

[qiao]

Context

I was trying to use Upstash SDK with Redis PubSub commands on Cloudflare Workers and was not able to receive the messages. After digging into the Upstash SDK source code and network packets I was able to find that the worker runtime was splitting the SSE message into multiple chunks in the middle of the line (which should be permitted), but the Upstash SDK was not properly handle this situation, causing the message to be silently dropped.

Problem

The streaming parser in the Upstash SDK processed each network chunk independently, splitting on newlines without buffering partial lines. This was incorrect because the stream reader is not guaranteed to emit chunks aligned with newlines (as in the Cloudflare case above). Chunks can be generated at arbitrary byte boundaries, including mid-line. When a message arrived split across multiple chunks, the parser attempted to process incomplete data, resulting in JSON parse errors and lost messages.

Example scenario:

A subscription receives the following message split across two chunks:

Chunk 1: data: {"id":"msg1","co
Chunk 2: ntent":"hello"}\n

Previous behavior:

• Attempts to parse {"id":"msg1","co → incomplete message, dropped.
• Attempts to parse ntent":"hello"} → incomplete message, dropped
• Message lost

Fix

Implemented line buffering to accumulate partial data between chunks:

1. Added a buffer variable to store incomplete lines
2. Modified the decoder to use { stream: true } for proper streaming text decoding (see TextDecoder.decode() documentation)
3. Updated line processing logic:
   • Append each chunk to the buffer
   • Split on \n to extract complete lines
   • Process only complete lines
   • Retain the last incomplete line in the buffer for the next chunk

The stream: true option indicates that additional data will follow in subsequent calls to decode(), which is essential for correctly handling multi-byte characters that may be split across chunk boundaries.

[CahidArda]

Hi @qiao,

Thanks for the report and the fix. We will continue under #1404 for the fix.

Thanks a lot!