demo/llama_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using InterSystems Vector Search with LlamaIndex\n",
    "\n",
    "In this notebook, we'll leverage the Vector Search capabilities available in [InterSystems IRIS 2024.1](https://www.intersystems.com/news/iris-vector-search-support-ai-applications/) and [InterSystems IRIS Cloud SQL](https://developer.intersystems.com/products/iris-cloud-sql-integratedml/), using the well-known [LlamaIndex](https://www.llamaindex.ai/) framework."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up the connection\n",
    "\n",
    "First, let's make sure we set up the connection to your InterSystems IRIS instance or Cloud SQL deployment. When targeting a Cloud SQL deployment, change the username and password to `SQLAdmin` and the corresponding password you chose when creating the deployment, and set the port to 443. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "username = 'demo'\n",
    "password = 'demo'\n",
    "hostname = os.getenv('IRIS_HOSTNAME', 'localhost')\n",
    "\n",
    "port = 1972 \n",
    "namespace = 'USER'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Securing the connection\n",
    "\n",
    "If the target you're connecting to requires secure connections, as is the case for Cloud SQL deployments, we need to supply a certificate and some additional settings to the driver. For Cloud SQL, you can download the certificate file from your deployment's details screen. Look for the button that says \"Get X.509 certificate\", and copy it into a local folder, such as `/usr/cert-demo/`. If you're running this notebook in a container, you can copy the certificate file into the container using the following command:\n",
    "\n",
    "```Shell\n",
    "docker cp ~/Downloads/certificateSQLaaS.pem iris-vector-search-jupyter-1:/usr/cert-demo/certificateSQLaaS.pem\n",
    "```\n",
    "\n",
    "Remember to also set the port to 443 in the cell above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No certificate file found, continuing with insecure connection\n"
     ]
    }
   ],
   "source": [
    "import ssl\n",
    "\n",
    "certificateFile = \"/usr/cert-demo/certificateSQLaaS.pem\"\n",
    "\n",
    "if (os.path.exists(certificateFile)):\n",
    "    print(\"Located SSL certficate at '%s', initializing SSL configuration\", certificateFile)\n",
    "    sslcontext = ssl.create_default_context(cafile=certificateFile)\n",
    "else:\n",
    "    print(\"No certificate file found, continuing with insecure connection\")\n",
    "    sslcontext = None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hello world!\n"
     ]
    }
   ],
   "source": [
    "from sqlalchemy import create_engine, text\n",
    "\n",
    "url = f\"iris://{username}:{password}@{hostname}:{port}/{namespace}\"\n",
    "\n",
    "engine = create_engine(url, connect_args={\"sslcontext\": sslcontext})\n",
    "with engine.connect() as conn:\n",
    "    print(conn.execute(text(\"SELECT 'hello world!'\")).first()[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Vectors using LlamaIndex\n",
    "\n",
    "In the following cell we'll leverage standard LlamaIndex components to read the files in the `/data/paul_graham/` directory and prepare them for creating embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First Document ID: 0d59fd19-5129-444b-9ad9-60bea9ffbae0\n"
     ]
    }
   ],
   "source": [
    "#from llama_index import SimpleDirectoryReader\n",
    "from llama_index.legacy import SimpleDirectoryReader\n",
    "\n",
    "documents = SimpleDirectoryReader(\"../data/paul_graham\").load_data()\n",
    "print(\"First Document ID:\", documents[0].doc_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setting up your OpenAI API key\n",
    "\n",
    "If you have an OpenAI subscription, use the following cell to pick up your OpenAI API key, and use `OpenAIEmbeddings()` in the cells below. \n",
    "\n",
    "Alternatively, you can skip this step and use a local embeddings model that's included in the libraries already imported, such as `HuggingFaceEmbeddings()`, `FastEmbeddings()`, or `FakeEmbeddings()` (for testing purposes!). Just comment / uncomment the corresponding lines in the cells further down the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "load_dotenv(override=True)\n",
    "\n",
    "if not os.environ.get(\"OPENAI_API_KEY\"):\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"OpenAI API Key:\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next cell sets up a local language model as the default to create the embeddings if you don't have an OpenAI key. LlamaIndex' default is to use OpenAI, so not running the following cell will assume you want to continue with OpenAI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.legacy import ServiceContext, set_global_service_context\n",
    "from llama_index.embeddings.langchain import LangchainEmbedding\n",
    "from langchain.embeddings import HuggingFaceEmbeddings\n",
    "from langchain.embeddings import FakeEmbeddings\n",
    "\n",
    "if not os.environ.get(\"OPENAI_API_KEY\"):\n",
    "    lc_embed_model = HuggingFaceEmbeddings(model_name=\"all-MiniLM-L6-v2\")\n",
    "    # lc_embed_model = FakeEmbeddings(size=1536)\n",
    "\n",
    "    # ServiceContext captures how vectors will be generated\n",
    "    service_context = ServiceContext.from_defaults(\n",
    "        embed_model=LangchainEmbedding(lc_embed_model), \n",
    "        llm=None\n",
    "    )\n",
    "\n",
    "    set_global_service_context(service_context)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll configure the VectorStore object that will be used to save our vectors in IRIS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index.legacy import StorageContext\n",
    "from llama_index.legacy.indices.vector_store import VectorStoreIndex\n",
    "from llama_iris import IRISVectorStore\n",
    "\n",
    "# StorageContext captures how vectors will be stored\n",
    "vector_store = IRISVectorStore.from_params(\n",
    "    connection_string = url,\n",
    "    table_name = \"paul_graham_essay\",\n",
    "    embed_dim = 1536,  # openai embedding dimension\n",
    "    engine_args = { \"connect_args\": {\"sslcontext\": sslcontext} }\n",
    ")\n",
    "storage_context = StorageContext.from_defaults(vector_store=vector_store)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now putting it all together: feeding the documents to our VectorStore."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a236540e09244ee7b76eac483ef524d2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Parsing nodes:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2a2c7ead5c7e41188df8ff7ba2dd5362",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating embeddings:   0%|          | 0/22 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "index = VectorStoreIndex.from_documents(\n",
    "    documents, \n",
    "    storage_context=storage_context, \n",
    "    show_progress=True, \n",
    ")\n",
    "query_engine = index.as_query_engine()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # If reconnecting to the vector store, use this: \n",
    "\n",
    "index = VectorStoreIndex.from_vector_store(vector_store=vector_store)\n",
    "storage_context = StorageContext.from_defaults(vector_store=vector_store)\n",
    "query_engine = index.as_query_engine()\n",
    "\n",
    "# Adding documents to existing index\n",
    "\n",
    "for d in documents:\n",
    "    index.insert(document=d, storage_context=storage_context)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = query_engine.query(\"What did the author do?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The author wrote essays and also started to think about other things they could work on.\n"
     ]
    }
   ],
   "source": [
    "import textwrap\n",
    "print(textwrap.fill(str(response), 100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AI was in the air in the mid 1980s.\n"
     ]
    }
   ],
   "source": [
    "response = query_engine.query(\"What happened in the mid 1980s?\")\n",
    "print(textwrap.fill(str(response), 100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}