Skip to content

Commit

Permalink
Fixed conflicts
Browse files Browse the repository at this point in the history
  • Loading branch information
milo157 committed Oct 30, 2023
2 parents d12b763 + b9a9bba commit c7cb914
Showing 1 changed file with 7 additions and 4 deletions.
11 changes: 7 additions & 4 deletions cerebrium/cortex/advanced-functionality/streaming.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@ title: "Streaming Endpoints"
---

Streaming allows users to stream live output from their models using server-sent event (SSE) streams. This works for Python objects which implement the
iterator protocol which is anything that essentially uses the 'yield' command in Python. You can return any content types as long as it is returned as a string
iterator protocol which is anything that essentially uses the 'yield' command in Python. You can return any content types as long as it is returned as a string

<Note>This feature is currently in beta and so if you would like to stream output, please replace '**predict**' in your endpoint url with '**stream**'</Note>
<Note>
This feature is currently in beta and so if you would like to stream output,
please replace '**predict**' in your endpoint url with '**stream**'
</Note>

Let us see how we can implement a simple example below:

Expand All @@ -16,6 +19,6 @@ def predict(item, run_id, logger):
time.sleep(1)
```

Once you deploy this code snippet and hit the stream endpoint, you will see the SSE events progressively appear. The latest Postman has great functionality to show this.
Once you deploy this code snippet and hit the stream endpoint, you will see the SSE events progressively appear. The latest Postman has great functionality to show this.

If you want to see an example of implementing this with Falcon-7b, please check out the example [here](https://github.com/CerebriumAI/examples/tree/master/7-streaming-endpoint)
If you want to see an example of implementing this with Falcon-7b, please check out the example [here](https://github.com/CerebriumAI/examples/tree/master/7-streaming-endpoint)

0 comments on commit c7cb914

Please sign in to comment.