Merge pull request #141 from CerebriumAI/michael/cer-2466-write-examp…

…le-on-streaming-documentation

Michael/cer 2466 write example on streaming documentation

cerebrium/cortex/advanced-functionality/streaming.mdx

@@ -0,0 +1,26 @@
+---
+title: "Streaming Endpoints"
+---
+
+Streaming allows users to stream live output from their models using server-sent event (SSE) streams. This works for Python objects which implement the
+iterator protocol which is anything that essentially uses the 'yield' command in Python. You can return any content types as long as it is returned as a string
+
+<Note>
+  This feature is currently in beta and so if you would like to stream output,
+  please replace '**predict**' in your endpoint url with '**stream**'
+</Note>
+
+Let us see how we can implement a simple example below:
+
+```python
+def predict(item, run_id, logger):
+    for i in range(10):
+        yield f"Number {i} "
+        time.sleep(1)
+```
+
+Once you deploy this code snippet and hit the stream endpoint, you will see the SSE events progressively appear. The latest Postman has great functionality to show this.
+
+![Streaming](/images/cortex/streaming-postman.png)
+
+If you want to see an example of implementing this with Falcon-7b, please check out the example [here](https://github.com/CerebriumAI/examples/tree/master/7-streaming-endpoint)

mint.json

@@ -78,11 +78,12 @@
             "cerebrium/cortex/advanced-functionality/persistent-storage",
             "cerebrium/cortex/advanced-functionality/using-secrets",
             "cerebrium/cortex/advanced-functionality/async-functions",
+            "cerebrium/cortex/advanced-functionality/config-files",
             "cerebrium/cortex/advanced-functionality/multi-gpu-inferencing",
             "cerebrium/cortex/advanced-functionality/long-running-tasks",
             "cerebrium/cortex/advanced-functionality/model-scaling",
             "cerebrium/cortex/advanced-functionality/faster-model-loading",
-            "cerebrium/cortex/advanced-functionality/config-files"
+            "cerebrium/cortex/advanced-functionality/streaming"
           ]
         },
         {