Piplines Install & Run (Guide)

[rootcli]

I created this little guide to help newbies Run pipelines, as it was a challenge for me to install and run pipelines. Hope it helps.

Pipelines Usage

Quick Start with Docker

Pipelines Repository Quick Guide

What to do Next?

1. Run the Pipelines container:

2. Login to Open WebUI:

   • In Open WebUI: Navigate to the Admin Panel > Settings > Connections > OpenAI API section.
   • Set the API URL to http://host.docker.internal:9099 and the API key to 0p3n-w3bu!. Your pipelines should now be active. IF localhost doesn't work use host.docker.internal

3. You can Download the examples folder (or individual files, as you wish) from the Pipelines repository:

`To Download the Examples Folder:

git clone https://github.com/open-webui/pipelines.git examples

Here we will perform an example on one file which is called wikipedia_pipeline.py you can direct download it to linux from the link below

curl -O https://raw.githubusercontent.com/open-webui/pipelines/c9766d630ab69549b06e994b39fb173a61bb82e8/examples/pipelines/integrations/wikipedia_pipeline.py

4. Place The downloaded file wikipedia_pipeline.py in pipelines Root Folder.

5. In the open webui interface Go to Admin Panel > Gear Icon in top right ⚙ > pipelines

   you should see Pipelines Valves click on the pulldown menu and select wikipedia_pipeline

3. Go to New Chat > From the Models pulldown Menu select ``

4. Now type a topic you wish to search from `wikipedia

done

[santosh0787]

I really like this feature. Can you please add some documentation on how to add tools functions? I added examples/filters/function_calling_filter_pipeline.py but it is added as filters. Can you tell how invoke these functions?
All I get is query is going to Wiki pipeline.

[c9482]

When I only select "Wikipedia Pipeline", I get nothing. I have to add another model to the chat to get a response and the response does not come from "Wikipedia Pipeline". I was expecting to get the latest entries from wikipedia. Perhaps I am still missing how this is supposed to work? See below.

[rootcli]

"no information found" means the piplines model didn't find information.
try "Sun" or "Moon" in the prompt

[c9482]

Thanks, I get the point.

[hblink]

and what is the point? i have the same problem!

[stevenfullman]

and what is the point? i have the same problem!

I think the point is that rather than write a "prompt" like "who won the world cup in 2022", the prompt is simply "world cup 2022". That's the only way I can get answers anyway

[delfireinoso]

Place The downloaded file wikipedia_pipeline.py in pipelines Root Folder.

I have the file wikipedia_pipeline.py
Where I find the Piplelines Root Folder?

[c9482]

It's easiest to install to install from Github URL. For manual installation, there's an empty "pipelines" folder in the repo. Or if you're using docker, mount a volume for /app/pipelines.

[delfireinoso]

I use Docker as it's easy on a Mac

To mount a volume I think that I need to modify the Docker install command for Pipelines

This should come on the install documentation

[delfireinoso]

I see that the Docker default install has a Pipelines Volume, I'll tried the url approch and it's fine and working

[c9482]

Still couple of unanswered questions about function calling filter pipeline:

1. How does it get activated? For example, function_calling_filter_pipeline example has get_current_time tool. I can't get it to fire. Is there a specific prompt that I need to use?
2. Are the changes to pipeline automatically detected and the pipeline is reloaded? Or do I have to restart the pipelines server?

Many thanks in advance for some clues.

[c9482]

This is great information, thanks! One other question: how are package dependencies resolved? It seems that only the Python file gets deployed. I am not a Python programmer so this is a bit of a mystery to me.

[atgehrhardt]

So I haven't built anything with external dependencies yet but I can test later. Looking through the pipelines code I'm not seeing a way to install dependencies or an automatic mechanism, so may not be implemented yet. You could dial into the docker and install them, but I think that may be problematic as conflicting packages would be an issue since it looks like pipelines aren't running in venvs.

I can't be sure, may be worth opening a discussion on the Pipelines github

[atgehrhardt]

Looks like this is addressed here: #65 (comment)

[justinh-rahb]

@atgehrhardt you can add a header to a pipeline such as this, that specifies the dependencies to install:

"""
title: MLX Pipeline
author: justinh-rahb
date: 2024-05-27
version: 1.1
license: MIT
description: A pipeline for generating text using Apple MLX Framework.
requirements: requests, mlx-lm, huggingface-hub
environment_variables: MLX_HOST, MLX_PORT, MLX_MODEL, MLX_STOP, MLX_SUBPROCESS, HUGGINGFACE_TOKEN
"""

from typing import List, Union, Generator, Iterator
from schemas import OpenAIChatMessage
import requests
import os
import subprocess
import logging
from huggingface_hub import login

...

[atgehrhardt]

Oh that's fantastic, thank you for the clarification!

[musicheardworldwide]

If you create tools or functions you want to be triggered, you have to create the filter as well. This took me until today to figure out, but your pipe is in the data, the filter is how it gets to where its going and how. Keep building up your tool pipelines or just tools, and make sure you have the python function filter up and accurate, it should just trigger whenever you say the right thing. also, dont forget every tool you make needs to have a call, so once you get that locked in it should be smooth...

i just had my model message me on icloud via tools. There's a LOT of potential with this

[justinh-rahb]

Am I correct in understanding that the difference between a filter and pipeline is that a filter is a middleware inserted between user input and sending it directly to an LLM, while a pipeline creates a "model" that appears in the dropdown?

Yes this is correct.

I see this snippet a lot in the examples, but what does it mean to be connected to a pipeline? My current usage is using the Azure pipeline as the model, so it would be great if I can configure filters to use that pipeline as well. Any clarification would be greatly appreciated!

# List target pipeline ids (models) that this filter will be connected to.
# If you want to connect this filter to all pipelines, you can set pipelines to ["*"]
pipelines: List[str] = []

This is how you determine which "models" or pipelines are acted upon by the filter. By default it will be all (*), or you can set model or pipeline IDs, comma-separated.

[nomppy]

Thanks very much for the clarification. So for example, if I want to implement function calling for Azure models, I would write the Azure pipeline id in the pipelines attribute for the function calling filter (or leave it as *), which will insert the function calling filter between the user's input and the Azure pipeline.

If I want the filter itself to use Azure OpenAI, e.g. to retrieve the function call parameters, is there a way to have a filter call a pipeline? This might be somewhat of a niche use. The other option I see is to just modify the filter directly to take in Azure keys instead.

Btw, what are blueprints and manifolds?

[justinh-rahb]

Blueprints are just part of the framework's boilerplate code, you shouldn't have any need to mess with these. Manifold is just a special pipeline class that provides more than one pipe back to the models array. So for example an LLM provider manifold will have a pipe for each model it provides.

[nomppy]

Given that the current function calling blueprint uses OpenAI as a provider, wouldn't I need to modify it if I want to use a different folder?

[TheDarkTrumpet]

Blueprints are just part of the framework's boilerplate code, you shouldn't have any need to mess with these. Manifold is just a special pipeline class that provides more than one pipe back to the models array. So for example an LLM provider manifold will have a pipe for each model it provides.

This is really helpful to have it explained. Was messing with pipelines and making raw OpenAI calls trying to figure out what's what in the repo. There's a good amount of documentation, but I almost feel like the Pipelines project should be treated more separately with their own set of documentation. That way it's more clear about things that specifically apply here. The Manifold really threw me for a loop. I thought it was a way for a way to compartmentalize the pipelines, and the manifold could call those pipelines as a collection.

[eladrave]

I installed the https://github.com/open-webui/pipelines/blob/main/examples/filters/function_calling_filter_pipeline.py

However, I can never get the weather part work.
I tried "weather in new york" , but it didn't use it.

I tested my access to openweathermap, and it worked fine:
curl -G "http://api.openweathermap.org/data/2.5/weather"
--data-urlencode "q=New York"
--data-urlencode "appid=MY_KEY"
--data-urlencode "units=imperial"

[saccodd]

does the current time work for you? for me none of them is working. I'm trying to use models provided through ollama (e.g. llama3.1:latest ) instead of the OpenAI endpoint and replacing it with http://host.docker.internal:11434/v1 or http://localhost:11434/v1 but nothing works

[PlebeiusGaragicus]

I cannot wait until more documentation and examples come out!!!

Right now I'm thinking I need a filter to track user token usage. When the user has used all their allotted tokens I need the AI model to respond with a payment request LNURL. I made my own webui scratch to do this, but now that Open WebUI includes this AWESOME custom extension ability I'm going to pivot towards this.

This could be THE feature I've been looking for in the ecosystem to build this out. Self-hosted, micropayment-enabled, custom LangGraph AI constructs for the rest of us...

[VfBfoerst]

If you want to run and connect the pipelines server with Podman , you need to slightly adjust the "run" command: host.docker.internal to host.containers.internal:

podman run --name pipelines -d -p 9099:9099 --add-host=host.containers.internal:host-gateway -v /path/on/host/pipelines:/app/pipelines --restart always ghcr.io/open-webui/pipelines:main

In the admin-Settings, you will need to do the same (http://host.containers.internal:9099):

[saccodd]

Trying to run https://github.com/open-webui/pipelines/blob/main/examples/filters/function_calling_filter_pipeline.py but still unsuccessful.

I am using Open WebUI with Bundled Ollama Support (see https://github.com/open-webui/open-webui?tab=readme-ov-file#installing-open-webui-with-bundled-ollama-support). Checking Docker logs I realised that Open WebUi was not able to connect to Ollama API (even if the downloaded models were working fine). So I reached the following setup that doesn't throw errors.

Launch container with:
docker run -d -p 4000:8080 --add-host=host.docker.internal:host-gateway -v ollama:/root/.ollama -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://host.docker.internal:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

Setup API endpoints as follows:

I installed the pipeline from Github URL and setup as follows:

If I use localhost instead of host.docker.internal as OpenAI API Base URL, still no error is thrown, but I can't get my models to use the pipeline.
Currently testing with llama3.1:latest and llama3-groq-tool-use:latest

[nomppy]

So the issue is what exactly? That you can't connect to Ollama or that the function calling filter isn't being triggered?

[saccodd]

function calling filter pipeline is not being triggered

[Urvesh71]

I also have executed Open Web UI in a docker container along with Ollama container running parallel. Apart from that I have executed the Pipeline container as well. And I followed the steps of adding http://host.docker.internal:9099 and 0p3n-w3bu! as API key. I varified successfully. But still I cannot see pipeline. In my Pipeline section, It still shows 'Pipelines Not Detected'.

[mythosofllama]

Hello,
I am currently trying to integrate Ollama into OpenWebUi with a pipeline.

My model also shows a connection to the pipeline, but gives the following error (when using Wikipedia_Pipeline.py from the GIT-pipelines-examples):

Ich benutze Ollama3.2:latest.

The API connections are as follows:

All connections to the servers can be established.

I run it on wsl with docker.

How can I fix this?

[Urvesh71]

Did you figure out any solution to fix your problem? I am also having the same problem but in other pipeline: text_to_sql_pipeline.py

[Urvesh71]

@mythosofllama check logs of pipeline container. I tried to implement the same and it works on my side

[F041]

I installed the wikipedia pipeline and the text to sql. I noticed I didn't see the latter in valves, so I repasted the URL for the installation multiple times. Result: it completed bricked OpenWebUI and in the docker logs I see

  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app

    raw_response = await run_endpoint_function(

                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function

    return await dependant.call(**values)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/app/backend/open_webui/main.py", line 1073, in get_models

    models = await get_all_models()

             ^^^^^^^^^^^^^^^^^^^^^^

  File "/app/backend/open_webui/main.py", line 929, in get_all_models

    ollama_models = await get_ollama_models()

                    ^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/app/backend/open_webui/apps/ollama/main.py", line 219, in get_all_models

    "models": merge_models_lists(

              ^^^^^^^^^^^^^^^^^^^

  File "/app/backend/open_webui/apps/ollama/main.py", line 196, in merge_models_lists

    for idx, model_list in enumerate(model_lists):

  File "/app/backend/open_webui/apps/ollama/main.py", line 221, in <lambda>

    lambda response: response["models"] if response else None, responses

And I have no idea on what to do

[F041]

but now what can I do to restore it before the bricking point?

[Urvesh71]

Delete the Image, Container and its volume and then Run the Docker compose file to rebuild the container

[F041]

the volume of just the pipeline? Or of all 3 services? (OWUI, Ollama, Pipelines)

[Urvesh71]

delete the Volume of entire container. Start with fresh container

[F041]

thanks. Do you have an easy backup solution so it won't happen again? I gave a look at FreeFileSync, Duplicati. I also saw other two docker based solutions, but I prefer GUI