Spring AI Watsonx.ai

🧪 Testing Status: Chat and Embeddings features are currently in testing phase using the watsonx.ai SaaS platform. Features and APIs may change without notice.

Spring AI Watsonx.ai provides Spring AI integration with IBM's Watsonx.ai platform, enabling developers to leverage powerful foundation models for chat, and embeddings in their applications.

Overview

IBM Watsonx.ai is an enterprise-ready AI platform that provides access to various foundation models including:

• Chat Models: IBM Granite, Meta Llama, Mistral AI, and other conversational AI models
• Embedding Models: IBM's embedding models for semantic search and similarity analysis

This integration brings these capabilities to Spring Boot applications through familiar Spring AI abstractions.

Features

• Chat Models: Support for multiple foundation models with streaming capabilities
• Embedding Models: Generate embeddings for semantic search and similarity analysis
• Moderation Models: Content moderation with HAP, PII, and Granite Guardian detectors
• Spring Boot Auto-configuration: Zero-configuration setup with Spring Boot
• Flexible Configuration: Runtime parameter overrides and multiple model configurations
• Function Calling: Connect LLMs with external tools and APIs
• Reactive Support: Built-in support for reactive programming with WebFlux

Quick Start

Prerequisites

1. Create an account at IBM Cloud
2. Set up a Watsonx.ai service instance
3. Generate API keys from the IBM Cloud console

Installation

Add the Spring AI Watsonx.ai starter to your project:

Maven:

<dependency>
    <groupId>org.springaicommunity</groupId>
    <artifactId>spring-ai-starter-model-watsonx-ai</artifactId>
    <version>1.0.0</version>
</dependency>

Gradle:

implementation 'org.springaicommunity:spring-ai-starter-model-watsonx-ai:1.0.0'

Configuration

Configure your application with Watsonx.ai credentials:

application.yml:

spring:
  ai:
    watsonx:
      ai:
        api-key: ${WATSONX_AI_API_KEY}
        url: ${WATSONX_AI_URL}
        project-id: ${WATSONX_AI_PROJECT_ID}

Environment Variables:

export WATSONX_AI_API_KEY=your_api_key_here
export WATSONX_AI_URL=https://us-south.ml.cloud.ibm.com
export WATSONX_AI_PROJECT_ID=your_project_id_here

Basic Usage

Chat Model

@RestController
public class ChatController {

    private final WatsonxAiChatModel chatModel;

    public ChatController(WatsonxAiChatModel chatModel) {
        this.chatModel = chatModel;
    }

    @GetMapping("/chat")
    public String chat(@RequestParam String message) {
        return chatModel.call(message);
    }

    @GetMapping("/chat/stream")
    public Flux<String> chatStream(@RequestParam String message) {
        return chatModel.stream(new Prompt(message))
            .map(response -> response.getResult().getOutput().getContent());
    }
}

Embedding Model

@RestController
public class EmbeddingController {

    private final WatsonxAiEmbeddingModel embeddingModel;

    public EmbeddingController(WatsonxAiEmbeddingModel embeddingModel) {
        this.embeddingModel = embeddingModel;
    }

    @GetMapping("/embed")
    public List<Double> embed(@RequestParam String text) {
        return embeddingModel.embed(text);
    }
}

Moderation Model

@RestController
public class ModerationController {

    private final WatsonxAiModerationModel moderationModel;

    public ModerationController(WatsonxAiModerationModel moderationModel) {
        this.moderationModel = moderationModel;
    }

    @PostMapping("/moderate")
    public ModerationResponse moderate(@RequestBody String text) {
        ModerationPrompt prompt = new ModerationPrompt(text);
        return moderationModel.call(prompt);
    }
}

Architecture

The Spring AI Watsonx.ai integration consists of three main modules:

Core Modules

• watsonx-ai-core: Core implementation with API clients and model classes
• spring-ai-autoconfigure-model-watsonx-ai: Spring Boot auto-configuration
• spring-ai-starter-model-watsonx-ai: Spring Boot starter for easy integration

Key Components

spring-ai-watsonx-ai/
├── watsonx-ai-core/
│   ├── WatsonxAiChatModel        # Chat model implementation
│   ├── WatsonxAiEmbeddingModel   # Embedding model implementation
│   ├── WatsonxAiModerationModel  # Content moderation implementation
│   └── WatsonxAiAuthentication   # IBM Cloud IAM authentication
├── spring-ai-autoconfigure-model-watsonx-ai/
│   └── Auto-configuration classes
└── spring-ai-starter-model-watsonx-ai/
    └── Starter dependencies

Supported Models

A comprehensive list of supported models under the watsonx.ai platform: watsonx.ai Supported Models

Configuration Options

Chat Model Configuration

spring:
  ai:
    watsonx:
      ai:
        chat:
          options:
            model: ibm/granite-13b-chat-v2
            temperature: 0.7
            max-new-tokens: 1024
            top-p: 1.0
            top-k: 50
            repetition-penalty: 1.0

Embedding Model Configuration

spring:
  ai:
    watsonx:
      ai:
        embedding:
          options:
            model: ibm/slate-125m-english-rtrvr
            parameters:
                truncate-input-tokens: true
                return-options:
                    input-text: false

Advanced Features

Function Calling

Connect your LLMs with external tools and APIs:

@Bean
@Description("Get current weather information")
public Function<WeatherRequest, WeatherResponse> getCurrentWeather() {
    return request -> {
        // Implementation to fetch weather data
        return new WeatherResponse(25.0, "sunny", request.location());
    };
}

Multiple Model Configurations

Configure different models for different use cases:

@Configuration
public class MultiModelConfiguration {

    @Bean("creativeChatModel")
    public WatsonxAiChatModel creativeChatModel(WatsonxAiChatApi chatApi) {
        return new WatsonxAiChatModel(chatApi,
            WatsonxAiChatOptions.builder()
                .withModel("meta-llama/llama-3-70b-instruct")
                .withTemperature(1.2)
                .build());
    }
}

Reactive Streaming

Built-in support for reactive programming:

@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ServerSentEvent<String>> streamResponse(@RequestParam String prompt) {
    return chatModel.stream(new Prompt(prompt))
        .map(response -> response.getResult().getOutput().getContent())
        .map(content -> ServerSentEvent.<String>builder().data(content).build());
}

Moderation Model Configuration

The moderation model provides content safety detection using Watsonx.ai's text detection API with multiple detector types:

Available Detectors:

• HAP (Hate, Abuse, Profanity): Detects hate speech, abusive language, and profanity
• PII (Personally Identifiable Information): Identifies sensitive personal information like emails, phone numbers, addresses
• Granite Guardian: IBM's comprehensive content moderation detector for harmful content

spring:
  ai:
    watsonx:
      ai:
        moderation:
          version: "2025-10-01"
          options:
            # HAP detector with 0.75 threshold (0.0-1.0)
            hap:
              threshold: 0.75
            # PII detector with 0.8 threshold
            pii:
              threshold: 0.8
            # Granite Guardian detector with 0.6 threshold
            granite-guardian:
              threshold: 0.6

Response Analysis:

// Check if content was flagged
boolean isFlagged = response.getResult().getOutput().getResults().get(0).isFlagged();

// Get category scores
CategoryScores scores = response.getResult().getOutput().getResults().get(0).getCategoryScores();
double hateScore = scores.getHate();
double harassmentScore = scores.getHarassment();

// Access Watsonx-specific metadata
WatsonxAiModerationResponseMetadata metadata =
    (WatsonxAiModerationResponseMetadata) response.getMetadata();

// Get detection positions and details
List<Map<String, Object>> detections = metadata.getDetections();
for (Map<String, Object> detection : detections) {
    String detectionType = (String) detection.get("detectionType");
    String text = (String) detection.get("text");
    Float score = (Float) detection.get("score");
    Integer start = (Integer) detection.get("start");
    Integer end = (Integer) detection.get("end");
}

Examples

Customer Support Chatbot

@Service
public class CustomerSupportService {

    private final WatsonxAiChatModel chatModel;

    public String handleQuery(String customerId, String query) {
        var options = WatsonxAiChatOptions.builder()
            .withModel("ibm/granite-13b-chat-v2")
            .withTemperature(0.3)
            .withFunction("getOrderStatus")
            .withFunction("createSupportTicket")
            .build();

        return chatModel.call(new Prompt(buildContextualPrompt(customerId, query), options));
    }
}

Document Analysis

@Service
public class DocumentAnalysisService {

    private final WatsonxAiChatModel chatModel;
    private final WatsonxAiEmbeddingModel embeddingModel;

    public DocumentAnalysis analyzeDocument(String content) {
        // Generate summary
        String summary = chatModel.call("Summarize: " + content);
        
        // Generate embeddings for similarity search
        List<Double> embeddings = embeddingModel.embed(content);
        
        return new DocumentAnalysis(summary, embeddings);
    }
}

Documentation

For comprehensive documentation, examples, and API reference, visit:

• Full Documentation
• Getting Started Guide
• Chat Models
• Embedding Models
• Configuration
• Examples

Building from Source

Prerequisites

• Java 17 or later
• Maven 3.8.4 or later

Build

git clone https://github.com/spring-ai-community/spring-ai-watsonx-ai.git
cd spring-ai-watsonx-ai
mvn clean install

Run Tests

mvn test

Build Documentation

cd docs
mvn clean package

Contributing

We welcome contributions! Please see our Contributing Guide for details on:

• Code of Conduct
• Development setup
• Submitting pull requests
• Reporting issues

Development Setup

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass
6. Submit a pull request

Community

• GitHub Discussions - Ask questions and share ideas
• Issues - Report bugs and request features
• Mailing List - Stay updated with announcements

License

This project is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Acknowledgments

• Spring AI - The foundational framework
• IBM Watsonx.ai - The AI platform
• Spring Community - The vibrant community