Model Context Protocol

[louis-jan]

How MCP works

MCP (Model Context Protocol) is an open protocol that enables secure, controlled interactions between AI applications and local or remote resources. Let’s break down how it works, then look at how we’ll use it in this guide.

General Architecture

At its core, MCP follows a client-server architecture where a host application can connect to multiple servers:

flowchart LR
    subgraph "Your Computer"
        Host["MCP Host\n(Claude, IDEs, Tools)"]
        S1["MCP Server A"]
        S2["MCP Server B"]
        S3["MCP Server C"]

        Host <-->|"MCP Protocol"| S1
        Host <-->|"MCP Protocol"| S2
        Host <-->|"MCP Protocol"| S3

        S1 <--> R1[("Local\nResource A")]
        S2 <--> R2[("Local\nResource B")]
    end

    subgraph "Internet"
        S3 <-->|"Web APIs"| R3[("Remote\nResource C")]
    end

Loading

• MCP Hosts: Programs like Jan Desktop, IDEs, or AI tools that want to access resources through MCP
• MCP Clients: Protocol clients that maintain 1:1 connections with servers
• MCP Servers: Lightweight programs that each expose specific capabilities through the standardized Model Context Protocol
• Local Resources: Your computer's resources (databases, files, services) that MCP servers can securely access
• Remote Resources: Resources available over the internet (e.g., through APIs) that MCP servers can connect to

Source: Quickstart - Model Context Protocol

In This Discussion

To demonstrate how MCP works in practice, let’s look at how Jan Desktop integrates MCP servers. Specifically, we’ll focus on a setup using @smithery/sdk, a library that simplifies MCP implementation.

flowchart LR
    subgraph "Your Computer"
        direction LR
        Jan["Jan Desktop"]
        SDK["@smithery/sdk"]
        Extension["App Extension"]
		MCP["MCP Server"]

		Jan <-->|"Extension Handlers"| Extension
        Extension <-->|"Tool Handlers"| SDK
		SDK <--> |"MCP Protocol\n(Queries & Results)"| MCP
        MCP <-->|"Web Search\n(API)"| Internet
    end

Loading

Workflow
1. Jan Desktop: Acts as the host application, initiating queries and managing user interactions.
2. App Extension: A custom module extending Jan’s capabilities, serving as a bridge to external tools.
3. @smithery/sdk: A library that handles MCP communication, translating requests from Jan into standardized MCP queries.
4. MCP Server: The endpoint that executes specific tasks (e.g., web searches, database lookups) and returns results to Jan.
5. Remote Resources: MCP servers can connect to APIs or other internet resources to retrieve data.

Why Use MCP with Jan?
• Standardization: MCP provides a consistent way to interact with various resources, making integration smoother.
• Flexibility: Connect Jan to any resource that MCP servers expose, from local files to cloud-based APIs.
• Security: Fine-grained control ensures only authorized resources are accessed.
• Scalability: Easily extend Jan’s capabilities by adding more MCP servers without modifying core functionality.

Coming Up
In the next sections, we’ll dive deeper into setting up @smithery/sdk, configuring MCP servers, and creating a seamless interaction flow in Jan Desktop.

Build the Extension Tool

Let’s use the Assistant extension as an example, since it serves as an implementation tool for extensions.

1. Create an extension tool:

import {
  AssistantTool,
  InferenceTool,
  MessageRequest,
} from '@janhq/core'

// Define new extension tool class
export class MCPTool extends InferenceTool {
  name: string = 'model-context-protocol'


  // Process new messages (intercept, transform messages payload before sending out to destination LLM)
  async process(
    data: MessageRequest,
    tool?: AssistantTool
  ): Promise<MessageRequest> {
    // Process message request here

	// START: See example below
	// transformedData
	// END: See example below
    return Promise.resolve(transformedData)
  }
}

2. Process messages using @smithery/sdk

import { Connection } from "@smithery/sdk"
import { OpenAIHandler } from "@smithery/sdk/openai"
import * as exa from "@smithery/mcp-exa"
import { OpenAI } from "openai"

const openai = new OpenAI()
const connection = await Connection.connect({
  exa: {
    server: exa.createServer({
      apiKey: "KEY_GOES_HERE",
    }),
  },
})

// Example messages
let messages: ChatCompletionMessageParam[] = [
  {
    role: "user",
    content: "In 2024, did OpenAI release GPT-5?",
  },
]
const handler = new OpenAIHandler(connection)

while (!isDone) {
  const response = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages,
    tools: await handler.listTools(),
  })
  // Handle tool calls
  const toolMessages = await handler.call(response)

  // Append new messages
  messages.push(response.choices[0].message)
  messages.push(...toolMessages)
  isDone = toolMessages.length === 0
}

// const transformedData = messages

Source: GitHub - smithery-ai/typescript-sdk: Smithery easily connects language models to Model Context Protocols, allowing you to build agentic flows that use resources and tools without being overwhelmed by JSON schemas.

3 Then the new tool code base will be something like this:

// Define new extension tool class
export class MCPTool extends InferenceTool {
  name: string = 'model-context-protocol'

  openai = new OpenAI()
  connection = await Connection.connect({
    exa: {
      server: exa.createServer({
        apiKey: "KEY_GOES_HERE",
      }),
    },
  })

  // Process new messages (intercept, transform messages payload before sending out to destination LLM)
  async process(
    data: MessageRequest,
    tool?: AssistantTool
  ): Promise<MessageRequest> {
    // Process message request here
		
    // START: See example below
    const handler = new OpenAIHandler(this.connection)
    let messages = data.messages

    while (!isDone) {
        const response = await this.openai.chat.completions.create({
        model: "gpt-4o-mini",
        messages,
        tools: await handler.listTools(),
        })
        // Handle tool calls
        const toolMessages = await handler.call(response)
	
         // Append new messages
        messages.push(response.choices[0].message)
        messages.push(...toolMessages)
        isDone = toolMessages.length === 0
    }
    // END: See example below
    return Promise.resolve({
        ...data,
        messages,
    })
  }
}

4. Register new tool in Assistant Extension

export default class JanAssistantExtension extends AssistantExtension {
	onLoad() {
		ToolManager.instance().register(new MCPTool())
	}
}

5. Define tool in assistant.json (this is not required when using the prebuilt extension that is included with the discussion)

{
   type: 'model-context-protocol',
   enabled: true,
   settings: {},
}

6. Build the extension into a packaged tgz file

Run in Jan

We use this extension file in the demo screenshots provided below.

Attachments:
This extension TGZ file was built from feat/mcp-assistant-extension
janhq-assistant-extension-1.0.1.tgz

1. Install the updated extension
   

2. Setting APIs
   

3. Disable Experimental Feature
   There is a known issue where Jan would ignore other tools and enable retrieval tool only when toggle it on. That will be fixed soon from Jan.
   

4. Create new thread
   Previous threads will have assistant’s tools defined in thread.json. To apply new tool changes, create a new thread.

5. Voila
   

That is just an example

The example above is not an official MCP implementation of Jan, which is more about demonstrating how simple users can extend Jan’s features using its extension framework.

It's not a good implementation since:

1. This is an example, so there are no code changes from Jan, but it’s scoped in extensions.
2. This example requires OpenAI client setup where it should make use of running LLM model. (in-progress)
3. Extension tools are hard coded on the Frontend (see retrieval and experimental disable step)
4. Tool use should be available across engines, from local to remote. We are moving engines to cortex.cpp server, which mean MCP implementation could be there as well to utilize running engine properly.
5. It's scoped at Javascript, moving to cortex.cpp with multiple sidecar runtimes support would be great, users can extend it using both python or javascript.

We are still working on it

Please stay tuned and experiment with the extension update mentioned above in the meantime. We will update Jan and its backend to ensure the features are properly built.