Skip to content
@bentoml

BentoML

The easiest way to build fast and reliable AI serving systems

Welcome to BentoML 👋 Twitter Follow Slack

github banner

What's cooking? 👩‍🍳

🍱 BentoML: The Unified Serving Framework for AI Systems

BentoML is a Python library for building online serving systems optimized for AI apps and model inference. It supports serving any model format/runtime and custom Python code, offering the key primitives for serving optimizations, task queues, batching, multi-model chains, distributed orchestration, and multi-GPU serving.

🦾 OpenLLM: Self-hosting Large Language Models Made Easy

Run any open-source LLMs (Llama 3.1, Qwen2, Phi3 and more) or custom models as OpenAI-compatible APIs with a single command. It features a built-in chat UI, state-of-the-art inference performance, and a simplified workflow for production-grade cloud deployment.

☁️ BentoCloud: Fast and scalable infrastructure for building and scaling with BentoML on the cloud

BentoCloud is the complete platform for enterprise AI teams to build and scale Compound AI systems. It brings cutting-edge AI infrastructure into your cloud environment, enabling AI teams to run inference with unparalleled efficiency, rapidly iterate on system design, and effortlessly scale in production with full observability.

Get in touch 💬

👉 Join our Slack community!

👀 Follow us on X @bentomlai and LinkedIn

📖 Read our blog

Pinned Loading

  1. BentoML BentoML Public

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    Python 7.1k 791

  2. OpenLLM OpenLLM Public

    Run any open-source LLMs, such as Llama, Gemma, as OpenAI compatible API endpoint in the cloud.

    Python 10k 636

Repositories

Showing 10 of 101 repositories
  • bentoml-comfyui Public

    BentoML extension for ComfyUI

    bentoml/bentoml-comfyui’s past year of commit activity
    Python 1 MIT 0 0 0 Updated Nov 8, 2024
  • ComfyUI-IDL Public

    ComfyUI Interface Definition Language

    bentoml/ComfyUI-IDL’s past year of commit activity
    Python 0 Apache-2.0 0 0 0 Updated Nov 8, 2024
  • BentoML Public

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    bentoml/BentoML’s past year of commit activity
    Python 7,132 Apache-2.0 791 157 15 Updated Nov 8, 2024
  • bentoml/openllm-models’s past year of commit activity
    Python 13 8 1 0 Updated Nov 8, 2024
  • yatai-image-builder Public

    🐳 Build OCI images for Bentos in k8s

    bentoml/yatai-image-builder’s past year of commit activity
    Go 14 9 4 10 Updated Nov 7, 2024
  • bentoml/yatai-common’s past year of commit activity
    Go 1 5 0 2 Updated Nov 7, 2024
  • bentoml/BentoComfyUI’s past year of commit activity
    Python 0 0 0 0 Updated Nov 6, 2024
  • OpenLLM Public

    Run any open-source LLMs, such as Llama, Gemma, as OpenAI compatible API endpoint in the cloud.

    bentoml/OpenLLM’s past year of commit activity
    Python 10,000 Apache-2.0 636 18 0 Updated Nov 5, 2024
  • BentoVLLM Public

    Self-host LLMs with vLLM and BentoML

    bentoml/BentoVLLM’s past year of commit activity
    Python 72 12 4 2 Updated Nov 5, 2024
  • bentoml/bentocloud-homepage-news’s past year of commit activity
    1 1 0 0 Updated Nov 1, 2024