Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/client/content/config/tabs/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ def display_settings():
if spring_ai_conf == "hybrid":
st.markdown(f"""
The current configuration combination of embedding and language models
is currently **not supported** for SpringAI.
is currently **not supported** for Spring AI and LangChain MCP templates.
- Language Model: **{ll_config.get("model", "Unset")}**
- Embedding Model: **{embed_config.get("model", "Unset")}**
""")
Expand Down
182 changes: 98 additions & 84 deletions src/client/mcp/rag/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ You need:
- Claude Desktop free

## Setup
With **[`uv`](https://docs.astral.sh/uv/getting-started/installation/)** installed, run the following commands in your current project directory `<PROJECT_DIR>/src/client/mcp/rag/`:
With **[`uv`](https://docs.astral.sh/uv/getting-started/installation/)** installed, run the following commands in your current project directory `<PROJECT_DIR>`:

```bash
uv init --python=3.11 --no-workspace
Expand All @@ -26,61 +26,52 @@ uv add mcp langchain-core==0.3.52 oracledb~=3.1 langchain-community==0.3.21 lang
```

## Export config
In the **AI Optimizer & Toolkit** web interface, after tested a configuration, in `Settings/Client Settings`:
In the **AI Optimizer & Toolkit** web interface, after tested a configuration, in `Configuration/Settings/Client Settings`:

![Client Settings](./images/export.png)

* select the checkbox `Include Sensitive Settings`
* press button `Download Settings` to download configuration in the project directory: `src/client/mcp/rag` as `optimizer_settings.json`.
* in `<PROJECT_DIR>/src/client/mcp/rag/rag_base_optimizer_config_mcp.py` change filepath with the absolute path of your `optimizer_settings.json` file.
* select the checkbox `Include Sensitive Settings`.
* press button `Download LangchainMCP` to download an VectorSearch MCP Agent built on current configuration.
* unzip the file in a `<PROJECT_DIR>` dir.


## Standalone client
There is a client that you can run without MCP via commandline to test it:
There is a client that you can run without MCP via command-line to test it:

```bash
uv run rag_base_optimizer_config.py "[YOUR_QUESTION]"
```
In `rag_base_optimizer_config_mcp.py`:

## Quick test via MCP "inspector"
## Claude Desktop setup

* Run the inspector:
Claude Desktop, in free version, not allows to connect remote server. You can overcome, for testing purpose only, with a proxy library called `mcp-remote`. These are the options.
If you have already installed Node.js v20.17.0+, it should work.

```bash
npx @modelcontextprotocol/inspector uv run rag_base_optimizer_config_mcp.py
```
* In **Claude Desktop** application, in `Settings/Developer/Edit Config`, get the `claude_desktop_config.json` to

* connect to the port `http://localhost:6274/` with your browser on the link printed, like in the following example:
```bash
..
Open inspector with token pre-filled:
http://localhost:6274/?MCP_PROXY_AUTH_TOKEN=cb2ef7521aaf2050ad9620bfb5e5df42dc958889e6e99ce4e9b18003eb93fffd
..
```

* setup the `Inspector Proxy Address` with `http://127.0.0.1:6277`
* test the tool developed.
* Set **remote sse** execution:

add the references to the local MCP server for RAG in the `<PROJECT_DIR>`:
```json
{
"mcpServers": {
...
,
"rag":{
"command": "npx",
"args": [
"mcp-remote",
"http://127.0.0.1:9090/sse"
]
}
}
}
```

## Claude Desktop setup

* In **Claude Desktop** application, in `Settings/Developer/Edit Config`, get the `claude_desktop_config.json` to add the references to the local MCP server for RAG in the `<PROJECT_DIR>/src/client/mcp/rag/`:
```json
{
"mcpServers": {
...
,
"rag":{
"command":"bash",
"args":[
"-c",
"source <PROJECT_DIR>/src/client/mcp/rag/.venv/bin/activate && uv run <PROJECT_DIR>/src/client/mcp/rag/rag_base_optimizer_config_mcp.py"
]
}
}
}
```
* In **Claude Desktop** application, in `Settings/General/Claude Settings/Configure`, under `Profile` tab, update fields like:

- `Full Name`
- `What should we call you`

Expand All @@ -94,8 +85,10 @@ Show the rag_tool message as-is, without modification.
```
This will impose the usage of `rag_tool` in any case.

**NOTICE**: If you prefer, in this agent dashboard or any other, you could setup a message in the conversation with the same content of `Instruction` to enforce the LLM to use the rag tool as well.

* Start MCP server in another shell in <PROJECT_DIR> with:
```bash
uv run rag_base_optimizer_config_mcp.py
```
* Restart **Claude Desktop**.

* You will see two warnings on rag_tool configuration: they will disappear and will not cause any issue in activating the tool.
Expand All @@ -106,38 +99,45 @@ This will impose the usage of `rag_tool` in any case.

If the question is related to the knowledge base content stored in the vector store, you will have an answer based on that information. Otherwise, it will try to answer considering information on which has been trained the LLM o other tools configured in the same Claude Desktop.

* **Optional**: for a **local stdio** execution, without launching the MCP Server:

* Add the references to the local MCP server for RAG in the `<PROJECT_DIR>`:
```json
{
"mcpServers": {
...
,
"rag":{
"command":"bash",
"args":[
"-c",
"source <PROJECT_DIR>/.venv/bin/activate && uv run <PROJECT_DIR>/rag_base_optimizer_config_mcp.py"
]
}
}
}
```
* Set `Local` with `Remote client` line in `<PROJECT_DIR>/rag_base_optimizer_config_mcp.py`:

## Make a remote MCP server the RAG Tool

In `rag_base_optimizer_config_mcp.py`:

* Update the absolute path of your `optimizer_settings.json`. Example:

```python
rag.set_optimizer_settings_path("/Users/cdebari/Documents/GitHub/ai-optimizer-mcp-export/src/client/mcp/rag/optimizer_settings.json")
```
```python
#mcp = FastMCP("rag", port=9090) #Remote client
mcp = FastMCP("rag") #Local
```

* Substitute `Local` with `Remote client` line:
* Substitute `stdio` with `sse` line of code:
```python
mcp.run(transport='stdio')
#mcp.run(transport='sse')
```

```python
#mcp = FastMCP("rag", port=9001) #Remote client
mcp = FastMCP("rag") #Local
```

* Substitute `stdio` with `sse` line of code:
```python
mcp.run(transport='stdio')
#mcp.run(transport='sse')
```
## Alternative way for a quick test: MCP "inspector"

* Start MCP server in another shell with:
* Start MCP server in another shell in <PROJECT_DIR> with:
```bash
uv run rag_base_optimizer_config_mcp.py
```


## Quick test

* Run the inspector:

```bash
Expand All @@ -148,38 +148,52 @@ npx @modelcontextprotocol/inspector

* set the Transport Type to `SSE`

* set the `URL` to `http://localhost:9001/sse`
* set the `URL` to `http://localhost:9090/sse`

* test the tool developed.



## Claude Desktop setup for remote/local server
Claude Desktop, in free version, not allows to connect remote server. You can overcome, for testing purpose only, with a proxy library called `mcp-remote`. These are the options.
If you have already installed Node.js v20.17.0+, it should work:
**Optional:** run with local **stdio** protocol
* Set as shown before the protolo to run locally in `<PROJECT_DIR>/rag_base_optimizer_config_mcp.py`:

* replace `rag` mcpServer, setting in `claude_desktop_config.json`:
```json
{
"mcpServers": {
"remote": {
"command": "npx",
"args": [
"mcp-remote",
"http://127.0.0.1:9001/sse"
]
}
}
}
```
* Set `Local` with `Remote client` line:

```python
#mcp = FastMCP("rag", port=9090) #Remote client
mcp = FastMCP("rag") #Local
```

* Substitute `stdio` with `sse` line of code:
```python
mcp.run(transport='stdio')
#mcp.run(transport='sse')
```

* Run the inspector:

```bash
npx @modelcontextprotocol/inspector uv run rag_base_optimizer_config_mcp.py
```

* connect to the port `http://localhost:6274/` with your browser on the link printed, like in the following example:
```bash
..
Open inspector with token pre-filled:
http://localhost:6274/?MCP_PROXY_AUTH_TOKEN=cb2ef7521aaf2050ad9620bfb5e5df42dc958889e6e99ce4e9b18003eb93fffd
..
```
* restart Claude Desktop.

* setup the `Inspector Proxy Address` with `http://127.0.0.1:6277`
* test the tool developed.



**NOTICE**: If you have any problem running, check the logs if it's related to an old npx/nodejs version used with mcp-remote library. Check with:
```bash
nvm -list
```
if you have any other versions available than the default. It could happen that Claude Desktop uses the older one. Try to remove any other nvm versions available to force the use the only one avalable, at minimum v20.17.0+.

* restart and test as remote server


* restart and test as remote server
Binary file modified src/client/mcp/rag/images/export.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
84 changes: 55 additions & 29 deletions src/client/mcp/rag/optimizer_utils/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,65 +16,91 @@
import oracledb

import logging

logging.basicConfig(level=logging.INFO)

logger = logging.getLogger(__name__)
logging.basicConfig(
level=logging.INFO,
format="%(name)s - %(levelname)s - %(message)s"
)

def get_llm(data):
logging.info("llm data:")
logging.info(data["user_settings"]["ll_model"]["model"])
logger.info("llm data:")
logger.info(data["client_settings"]["ll_model"]["model"])
llm = {}
llm_config = data["ll_model_config"][data["user_settings"]["ll_model"]["model"]]
models_by_id = {m["id"]: m for m in data.get("model_configs", [])}
llm_config= models_by_id.get(data["client_settings"]["ll_model"]["model"])
logger.info(llm_config)
provider = llm_config["provider"]
url = llm_config["url"]
api_key = llm_config["api_key"]
model = data["user_settings"]["ll_model"]["model"]
logging.info(f"CHAT_MODEL: {model} {provider} {url} {api_key}")
model = data["client_settings"]["ll_model"]["model"]
logger.info(f"CHAT_MODEL: {model} {provider} {url} {api_key}")
if provider == "ollama":
# Initialize the LLM
llm = OllamaLLM(model=model, base_url=url)
logger.info("Ollama LLM created")
elif provider == "openai":
llm = llm = ChatOpenAI(model=model, api_key=api_key)
logger.info("OpenAI LLM created")
return llm


def get_embeddings(data):
embeddings = {}
model = data["user_settings"]["vector_search"]["model"]
provider = data["embed_model_config"][model]["provider"]
url = data["embed_model_config"][model]["url"]
api_key = data["embed_model_config"][model]["api_key"]
logging.info(f"EMBEDDINGS: {model} {provider} {url} {api_key}")
logger.info("getting embeddings..")
model = data["client_settings"]["vector_search"]["model"]
logger.info(f"embedding model: {model}")
models_by_id = {m["id"]: m for m in data.get("model_configs", [])}
model_params= models_by_id.get(model)
provider = model_params["provider"]
url = model_params["url"]
api_key = model_params["api_key"]

logger.info(f"Embeddings Model: {model} {provider} {url} {api_key}")
embeddings = {}
if provider == "ollama":
embeddings = OllamaEmbeddings(model=model, base_url=url)
elif provider == "openai":
logging.info("BEFORE create embbedding")
logger.info("Ollama Embeddings connection successful")
elif (provider == "openai") or (provider == "openai_compatible"):
embeddings = OpenAIEmbeddings(model=model, api_key=api_key)
logging.info("AFTER create emebdding")
logger.info("OpenAI embeddings connection successful")
return embeddings


def get_vectorstore(data, embeddings):
config = data["database_config"][data["user_settings"]["database"]["alias"]]
logging.info(config)
db_alias=data["client_settings"]["database"]["alias"]


conn23c = oracledb.connect(user=config["user"], password=config["password"], dsn=config["dsn"])
db_by_name = {m["name"]: m for m in data.get("database_configs", [])}
db_config= db_by_name.get(db_alias)

table_alias=data["client_settings"]["vector_search"]["alias"]
model=data["client_settings"]["vector_search"]["model"]
chunk_size=str(data["client_settings"]["vector_search"]["chunk_size"])
chunk_overlap=str(data["client_settings"]["vector_search"]["chunk_overlap"])
distance_metric=data["client_settings"]["vector_search"]["distance_metric"]
index_type=data["client_settings"]["vector_search"]["index_type"]

logging.info("DB Connection successful!")
metric = data["user_settings"]["vector_search"]["distance_metric"]
db_table=(table_alias+"_"+model+"_"+chunk_size+"_"+chunk_overlap+"_"+distance_metric+"_"+index_type).upper().replace("-", "_")
logger.info(f"db_table:{db_table}")


user=db_config["user"]
password=db_config["password"]
dsn=db_config["dsn"]

logger.info(f"{db_table}: {user}/{password} - {dsn}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do NOT log the password.

conn23c = oracledb.connect(user=user, password=password, dsn=dsn)

logger.info("DB Connection successful!")
metric = data["client_settings"]["vector_search"]["distance_metric"]

dist_strategy = DistanceStrategy.COSINE
if metric == "COSINE":
dist_strategy = DistanceStrategy.COSINE
elif metric == "EUCLIDEAN":
dist_strategy = DistanceStrategy.EUCLIDEAN

a = data["user_settings"]["vector_search"]["vector_store"]
logging.info(f"{a}")
logging.info(f"BEFORE KNOWLEDGE BASE")
logging.info(embeddings)
knowledge_base = OracleVS(
conn23c, embeddings, data["user_settings"]["vector_search"]["vector_store"], dist_strategy
)

logger.info(embeddings)
knowledge_base = OracleVS(client=conn23c,table_name=db_table, embedding_function=embeddings, distance_strategy=dist_strategy)

return knowledge_base
Loading