Enhance YAML pipeline deployments using inputs / outputs fields

README.md

@@ -32,6 +32,7 @@ With Hayhooks, you can:
   - [Async Run API Method](#run_api_async)
   - [PipelineWrapper development with `overwrite` option](#pipelinewrapper-development-with-overwrite-option)
   - [Additional Dependencies](#additional-dependencies)
+- [Deploy a YAML Pipeline](#deploy-a-yaml-pipeline)
 - [Deploy an Agent](#deploy-an-agent)
 - [Support file uploads](#support-file-uploads)
 - [Run pipelines from the CLI](#run-pipelines-from-the-cli)
@@ -61,15 +62,13 @@ With Hayhooks, you can:
   - [Run Hayhooks Programmatically](#run-hayhooks-programmatically)
   - [Sharing code between pipeline wrappers](#sharing-code-between-pipeline-wrappers)
 - [Deployment Guidelines](#deployment-guidelines)
-- [Legacy Features](#legacy-features)
-  - [Deploy Pipeline Using YAML](#deploy-a-pipeline-using-only-its-yaml-definition)
 - [License](#license)
 
 ## Quick start with Docker Compose
 
 To quickly get started with Hayhooks, we provide a ready-to-use Docker Compose 🐳 setup with pre-configured integration with [open-webui](https://openwebui.com/).
 
-It's available [here](https://github.com/deepset-ai/hayhooks-open-webui-docker-compose).
+It's available in the [Hayhooks + Open WebUI Docker Compose repository](https://github.com/deepset-ai/hayhooks-open-webui-docker-compose).
 
 ## Quick start
 
@@ -162,8 +161,9 @@ CLI commands are basically wrappers around the HTTP API of the server. The full
 hayhooks run     # Start the server
 hayhooks status  # Check the status of the server and show deployed pipelines
 
-hayhooks pipeline deploy-files <path_to_dir>   # Deploy a pipeline using PipelineWrapper
-hayhooks pipeline deploy <pipeline_name>       # Deploy a pipeline from a YAML file
+hayhooks pipeline deploy-yaml <path_to_yaml>   # Deploy a pipeline from a YAML file (preferred)
+hayhooks pipeline deploy-files <path_to_dir>   # Deploy a pipeline using PipelineWrapper files
+hayhooks pipeline deploy <path_to_dir>         # Alias for deploy-files
 hayhooks pipeline undeploy <pipeline_name>     # Undeploy a pipeline
 hayhooks pipeline run <pipeline_name>          # Run a pipeline
 ```
@@ -195,7 +195,7 @@ The pipeline wrapper provides a flexible foundation for deploying Haystack pipel
 - Define custom execution logic with configurable inputs and outputs
 - Optionally expose OpenAI-compatible chat endpoints with streaming support for integration with interfaces like [open-webui](https://openwebui.com/)
 
-The `pipeline_wrapper.py` file must contain an implementation of the `BasePipelineWrapper` class (see [here](src/hayhooks/server/utils/base_pipeline_wrapper.py) for more details).
+The `pipeline_wrapper.py` file must contain an implementation of the `BasePipelineWrapper` class (see [BasePipelineWrapper source](src/hayhooks/server/utils/base_pipeline_wrapper.py) for more details).
 
 A minimal `PipelineWrapper` looks like this:
 
@@ -274,6 +274,8 @@ hayhooks pipeline deploy-files -n chat_with_website examples/pipeline_wrappers/c
 
 This will deploy the pipeline with the name `chat_with_website`. Any error encountered during development will be printed to the console and show in the server logs.
 
+Alternatively, you can deploy via HTTP: `POST /deploy_files` (CLI alias: `hayhooks pipeline deploy`).
+
 #### PipelineWrapper development with `overwrite` option
 
 During development, you can use the `--overwrite` flag to redeploy your pipeline without restarting the Hayhooks server:
@@ -318,6 +320,54 @@ Then, assuming you've installed the Hayhooks package in a virtual environment, y
 pip install trafilatura
 ```
 
+## Deploy a YAML Pipeline
+
+You can deploy a Haystack pipeline directly from its YAML definition using the preferred `/deploy-yaml` endpoint. This mode builds request/response schemas from the YAML-declared `inputs` and `outputs`.
+
+Note: You can also deploy YAML pipelines from the CLI with `hayhooks pipeline deploy-yaml`. Wrapper-based deployments continue to use `/deploy_files` or the CLI alias `hayhooks pipeline deploy`.
+
+Tip: You can obtain a pipeline's YAML from an existing `Pipeline` instance using `pipeline.dumps()`. See the [Haystack serialization docs](https://docs.haystack.deepset.ai/docs/serialization) for details.
+
+Requirements:
+
+- The YAML must declare both `inputs` and `outputs` fields so the API request/response schemas can be generated.
+- `inputs`/`outputs` entries map friendly names to pipeline component fields (e.g. `fetcher.urls`, `prompt.query`).
+
+Minimal example:
+
+```yaml
+# ... pipeline definition ...
+
+inputs:
+  urls:
+    - fetcher.urls
+  query:
+    - prompt.query
+outputs:
+  replies: llm.replies
+```
+
+CLI:
+
+```shell
+hayhooks pipeline deploy-yaml -n inputs_outputs_pipeline pipelines/inputs_outputs_pipeline.yml
+```
+
+Alternatively, you can deploy via HTTP: `POST /deploy-yaml`.
+
+If successful, the server exposes a run endpoint at `/{name}/run` with a request/response schema derived from the YAML IO. For example:
+
+```shell
+curl -X POST \
+  http://HAYHOOKS_HOST:HAYHOOKS_PORT/inputs_outputs_pipeline/run \
+  -H 'Content-Type: application/json' \
+  -d '{"urls": ["https://haystack.deepset.ai"], "query": "What is Haystack?"}'
+```
+
+Limitations:
+
+- YAML-deployed pipelines do not support OpenAI-compatible chat completion endpoints, so they cannot be used with Open WebUI. If you need chat completion/streaming, use a `PipelineWrapper` and implement `run_chat_completion` or `run_chat_completion_async` (see the OpenAI compatibility section below).
+
 ## Deploy an Agent
 
 Deploying a [Haystack Agent](https://docs.haystack.deepset.ai/docs/agents) is very similar to deploying a pipeline.
@@ -971,24 +1021,6 @@ We have some dedicated documentation for deployment:
 
 We also have some additional deployment guidelines, see [deployment_guidelines.md](docs/deployment_guidelines.md).
 
-### Legacy Features
-
-#### Deploy a pipeline using only its YAML definition
-
-**⚠️ This way of deployment is not maintained anymore and will be deprecated in the future**.
-
-We're still supporting the Hayhooks _former_ way to deploy a pipeline.
-
-The former command `hayhooks deploy` is now changed to `hayhooks pipeline deploy` and can be used to deploy a pipeline only from a YAML definition file.
-
-For example:
-
-```shell
-hayhooks pipeline deploy -n chat_with_website examples/pipeline_wrappers/chat_with_website/chat_with_website.yml
-```
-
-This will deploy the pipeline with the name `chat_with_website` from the YAML definition file `examples/pipeline_wrappers/chat_with_website/chat_with_website.yml`. You then can check the generated docs at `http://HAYHOOKS_HOST:HAYHOOKS_PORT/docs` or `http://HAYHOOKS_HOST:HAYHOOKS_PORT/redoc`, looking at the `POST /chat_with_website` endpoint.
-
 ### License
 
 This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.

src/hayhooks/cli/pipeline.py

@@ -38,23 +38,51 @@ def _deploy_with_progress(ctx: typer.Context, name: str, endpoint: str, payload:
         show_error_and_abort(f"Pipeline '[bold]{name}[/bold]' already exists! ⚠️")
 
 
-@pipeline.command()
-def deploy(
+@pipeline.command(name="deploy-yaml")
+def deploy_yaml(  # noqa: PLR0913
     ctx: typer.Context,
-    name: Annotated[Optional[str], typer.Option("--name", "-n", help="The name of the pipeline to deploy.")],
     pipeline_file: Path = typer.Argument(  # noqa: B008
-        help="The path to the pipeline file to deploy."
+        help="The path to the YAML pipeline file to deploy."
     ),
+    name: Annotated[Optional[str], typer.Option("--name", "-n", help="The name of the pipeline to deploy.")] = None,
+    overwrite: Annotated[
+        bool, typer.Option("--overwrite", "-o", help="Whether to overwrite the pipeline if it already exists.")
+    ] = False,
+    description: Annotated[
+        Optional[str], typer.Option("--description", help="Optional description for the pipeline.")
+    ] = None,
+    skip_mcp: Annotated[
+        bool, typer.Option("--skip-mcp", help="If set, skip MCP integration for this pipeline.")
+    ] = False,
+    save_file: Annotated[
+        bool,
+        typer.Option(
+            "--save-file/--no-save-file",
+            help="Whether to save the YAML under pipelines/{name}.yml on the server.",
+        ),
+    ] = True,
 ) -> None:
-    """Deploy a pipeline to the Hayhooks server."""
+    """Deploy a YAML pipeline using the preferred /deploy-yaml endpoint."""
     if not pipeline_file.exists():
         show_error_and_abort("Pipeline file does not exist.", str(pipeline_file))
 
     if name is None:
         name = pipeline_file.stem
 
-    payload = {"name": name, "source_code": pipeline_file.read_text()}
-    _deploy_with_progress(ctx=ctx, name=name, endpoint="deploy", payload=payload)
+    payload = {
+        "name": name,
+        "source_code": pipeline_file.read_text(),
+        "overwrite": overwrite,
+        "save_file": save_file,
+    }
+
+    if description is not None:
+        payload["description"] = description
+
+    # Always include skip_mcp flag (defaults to False)
+    payload["skip_mcp"] = skip_mcp
+
+    _deploy_with_progress(ctx=ctx, name=name, endpoint="deploy-yaml", payload=payload)
 
 
 @pipeline.command()
@@ -91,6 +119,10 @@ def deploy_files(
     _deploy_with_progress(ctx=ctx, name=name, endpoint="deploy_files", payload=payload)
 
 
+# Register alias: `deploy` -> `deploy-files`
+pipeline.command(name="deploy")(deploy_files)
+
+
 @pipeline.command()
 def undeploy(
     ctx: typer.Context,

src/hayhooks/server/app.py

@@ -11,12 +11,7 @@
 
 from hayhooks.server.logger import log
 from hayhooks.server.routers import deploy_router, draw_router, openai_router, status_router, undeploy_router
-from hayhooks.server.utils.deploy_utils import (
-    PipelineDefinition,
-    deploy_pipeline_def,
-    deploy_pipeline_files,
-    read_pipeline_files_from_dir,
-)
+from hayhooks.server.utils.deploy_utils import deploy_pipeline_files, deploy_pipeline_yaml, read_pipeline_files_from_dir
 from hayhooks.settings import APP_DESCRIPTION, APP_TITLE, check_cors_settings, settings
 
 
@@ -35,8 +30,7 @@ def deploy_yaml_pipeline(app: FastAPI, pipeline_file_path: Path) -> dict:
     with open(pipeline_file_path) as pipeline_file:
         source_code = pipeline_file.read()
 
-    pipeline_definition = PipelineDefinition(name=name, source_code=source_code)
-    deployed_pipeline = deploy_pipeline_def(app, pipeline_definition)
+    deployed_pipeline = deploy_pipeline_yaml(pipeline_name=name, source_code=source_code, app=app)
     log.info(f"Deployed pipeline from yaml: {deployed_pipeline['name']}")
 
     return deployed_pipeline

src/hayhooks/server/exceptions.py

@@ -10,6 +10,12 @@ class PipelineWrapperError(Exception):
     pass
 
 
+class PipelineYamlError(Exception):
+    """Exception for errors loading pipeline YAML."""
+
+    pass
+
+
 class PipelineModuleLoadError(Exception):
     """Exception for errors loading pipeline module."""
 
@@ -24,3 +30,9 @@ class PipelineNotFoundError(Exception):
     """Exception for errors when a pipeline is not found."""
 
     pass
+
+
+class InvalidYamlIOError(Exception):
+    """Exception for invalid or missing YAML inputs/outputs declarations."""
+
+    pass