gemini/getting-started/intro_genai_sdk.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ur8xi4C7S06n"
      },
      "outputs": [],
      "source": [
        "# Copyright 2024 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAPoU8Sm5E6e"
      },
      "source": [
        "# Getting started with Google Generative AI using the Gen AI SDK\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_genai_sdk.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_genai_sdk.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_genai_sdk.ipynb\">\n",
        "      <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/vertexai/v1/32px.svg\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_genai_sdk.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://upload.wikimedia.org/wikipedia/commons/9/91/Octicons-mark-github.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "\n",
        "<div style=\"clear: both;\"></div>\n",
        "\n",
        "<b>Share to:</b>\n",
        "\n",
        "<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_genai_sdk.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_genai_sdk.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_genai_sdk.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/53/X_logo_2023_original.svg\" alt=\"X logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_genai_sdk.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_genai_sdk.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
        "</a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "84f0f73a0f76"
      },
      "source": [
        "| | |\n",
        "|-|-|\n",
        "|Author(s) | [Eric Dong](https://github.com/gericdong)|"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvgnzT1CKxrO"
      },
      "source": [
        "## Overview\n",
        "\n",
        "The [Google Gen AI SDK](https://googleapis.github.io/python-genai/) provides a unified interface to Google's generative AI API services. This SDK simplifies the process of integrating generative AI capabilities into applications and services, enabling developers to leverage Google's advanced AI models for various tasks.\n",
        "\n",
        "In this tutorial, you learn about the key features of the Google Gen AI SDK for Python to help you get started with Google generative AI services and models including Gemini. You will complete the following tasks:\n",
        "\n",
        "- Install the Gen AI SDK\n",
        "- Connect to an API service\n",
        "- Send text prompts\n",
        "- Send multimodal prompts\n",
        "- Set system instruction\n",
        "- Configure model parameters\n",
        "- Configure safety filters\n",
        "- Start a multi-turn chat\n",
        "- Control generated output\n",
        "- Generate content stream\n",
        "- Send asynchronous requests\n",
        "- Count tokens and compute tokens\n",
        "- Use context caching\n",
        "- Function calling\n",
        "- Batch prediction\n",
        "- Get text embeddings\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "61RBz8LLbxCR"
      },
      "source": [
        "## Getting started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "No17Cw5hgx12"
      },
      "source": [
        "### Install Google Gen AI SDK\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tFy3H3aPgx12"
      },
      "outputs": [],
      "source": [
        "%pip install --upgrade --user --quiet google-genai pandas"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmWOrTJ3gx13"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "If you're running this notebook on Google Colab, run the cell below to authenticate your environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NyKGtVQjgx13"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EdvJRUWRNGHE"
      },
      "source": [
        "## Using Google Gen AI SDK\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "qgdSpVmDbdQ9"
      },
      "outputs": [],
      "source": [
        "import datetime\n",
        "\n",
        "from google import genai\n",
        "from google.genai.types import (\n",
        "    CreateBatchJobConfig,\n",
        "    CreateCachedContentConfig,\n",
        "    EmbedContentConfig,\n",
        "    FunctionDeclaration,\n",
        "    GenerateContentConfig,\n",
        "    Part,\n",
        "    SafetySetting,\n",
        "    Tool,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ve4YBlDqzyj9"
      },
      "source": [
        "## Connect to a Generative AI API service\n",
        "\n",
        "Google Gen AI APIs and models including Gemini are available in the following two API services:\n",
        "\n",
        "- **[Google AI for Developers](https://ai.google.dev/gemini-api/docs)**: Experiment, prototype, and deploy small projects.\n",
        "- **[Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/overview)**: Build enterprise-ready projects on Google Cloud.\n",
        "\n",
        "The Gen AI SDK provided an unified interface to these two API services. This notebook shows how to use the Gen AI SDK in Vertex AI."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eN9kmPKJGAJQ"
      },
      "source": [
        "### Vertex AI\n",
        "\n",
        "To start using Vertex AI, you must have a Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DF4l8DTdWgPY"
      },
      "source": [
        "#### Set Google Cloud project information\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "Nqwi-5ufWp_B"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "PROJECT_ID = \"[your-project-id]\"  # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
        "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
        "    PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
        "\n",
        "LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "T-tiytzQE0uM"
      },
      "outputs": [],
      "source": [
        "client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eXHJi5B6P5vd"
      },
      "source": [
        "## Choose a model\n",
        "\n",
        "For more information about all AI models and APIs on Vertex AI, see [Google Models](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models#gemini-models) and [Model Garden](https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "-coEslfWPrxo"
      },
      "outputs": [],
      "source": [
        "MODEL_ID = \"gemini-1.5-flash\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "37CH91ddY9kG"
      },
      "source": [
        "## Send text prompts\n",
        "\n",
        "Use the `generate_content` method to generate responses to your prompts. You can pass text to `generate_content`, and use the `.text` property to get the text content of the response."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6fc324893334"
      },
      "outputs": [],
      "source": [
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID, contents=\"What's the largest planet in our solar system?\"\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zurBcEcWhFc6"
      },
      "source": [
        "Optionally, you can display the response in markdown."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3PoF18EwhI7e"
      },
      "outputs": [],
      "source": [
        "from IPython.display import Markdown, display\n",
        "\n",
        "display(Markdown(response.text))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rZV2TY5Pa3Dd"
      },
      "source": [
        "## Send multimodal prompts\n",
        "\n",
        "You can include text, PDF documents, images, audio and video in your prompt requests and get text or code responses."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D3SI1X-JVMBj"
      },
      "outputs": [],
      "source": [
        "from PIL import Image\n",
        "import requests\n",
        "\n",
        "image = Image.open(\n",
        "    requests.get(\n",
        "        \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/meal.png\",\n",
        "        stream=True,\n",
        "    ).raw\n",
        ")\n",
        "\n",
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=[\n",
        "        image,\n",
        "        \"Write a short and engaging blog post based on this picture.\",\n",
        "    ],\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eN6wMdY1RSk3"
      },
      "source": [
        "You can also pass the file URL in `Part.from_uri` in the request to the model directly."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pG6l1Fuka6ZJ"
      },
      "outputs": [],
      "source": [
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=[\n",
        "        Part.from_uri(\n",
        "            file_uri=\"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/meal.png\",\n",
        "            mime_type=\"image/png\",\n",
        "        ),\n",
        "        \"Write a short and engaging blog post based on this picture.\",\n",
        "    ],\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "El1lx8P9ElDq"
      },
      "source": [
        "## Set system instruction\n",
        "\n",
        "[System instructions](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instruction-introduction) allow you to steer the behavior of the model. By setting the system instruction, you are giving the model additional context to understand the task, provide more customized responses, and adhere to guidelines over the user interaction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7A-yANiyCLaO"
      },
      "outputs": [],
      "source": [
        "system_instruction = \"\"\"\n",
        "  You are a helpful language translator.\n",
        "  Your mission is to translate text in English to French.\n",
        "\"\"\"\n",
        "\n",
        "prompt = \"\"\"\n",
        "  User input: I like bagels.\n",
        "  Answer:\n",
        "\"\"\"\n",
        "\n",
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=prompt,\n",
        "    config=GenerateContentConfig(\n",
        "        system_instruction=system_instruction,\n",
        "    ),\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hIJVEr0RQY8S"
      },
      "source": [
        "## Configure model parameters\n",
        "\n",
        "You can include parameter values in each call that you send to a model to control how the model generates a response. Learn more about [experimenting with parameter values](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/adjust-parameter-values)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d9NXP5N2Pmfo"
      },
      "outputs": [],
      "source": [
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=\"Tell me how the internet works, but pretend I'm a puppy who only understands squeaky toys.\",\n",
        "    config=GenerateContentConfig(\n",
        "        temperature=0.4,\n",
        "        top_p=0.95,\n",
        "        top_k=20,\n",
        "        candidate_count=1,\n",
        "        seed=5,\n",
        "        max_output_tokens=100,\n",
        "        stop_sequences=[\"STOP!\"],\n",
        "        presence_penalty=0.0,\n",
        "        frequency_penalty=0.0,\n",
        "    ),\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H9daipRiUzAY"
      },
      "source": [
        "## Configure safety filters\n",
        "\n",
        "The Gemini API provides safety filters that you can adjust across multiple filter categories to restrict or allow certain types of content. You can use these filters to adjust what's appropriate for your use case. See the [Configure safety filters](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-filters) page for details."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yPlDRaloU59b"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "    Write a list of 2 disrespectful things that I might say to the universe after stubbing my toe in the dark.\n",
        "\"\"\"\n",
        "\n",
        "safety_settings = [\n",
        "    SafetySetting(\n",
        "        category=\"HARM_CATEGORY_DANGEROUS_CONTENT\",\n",
        "        threshold=\"BLOCK_ONLY_HIGH\",\n",
        "    ),\n",
        "]\n",
        "\n",
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=prompt,\n",
        "    config=GenerateContentConfig(\n",
        "        safety_settings=safety_settings,\n",
        "    ),\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DpKKhHbx3CaJ"
      },
      "source": [
        "When you make a request to the model, the content is analyzed and assigned a safety rating. You can inspect the safety ratings of the generated content by printing out the model responses, as in this example:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7R7eyEBetsns"
      },
      "outputs": [],
      "source": [
        "print(response.candidates[0].safety_ratings)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "29jFnHZZWXd7"
      },
      "source": [
        "## Start a multi-turn chat\n",
        "\n",
        "The Gemini API enables you to have freeform conversations across multiple turns."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DbM12JaLWjiF"
      },
      "outputs": [],
      "source": [
        "system_instruction = \"\"\"\n",
        "  You are an expert software developer and a helpful coding assistant.\n",
        "  You are able to generate high-quality code in any programming language.\n",
        "\"\"\"\n",
        "\n",
        "chat = client.chats.create(\n",
        "    model=MODEL_ID,\n",
        "    config=GenerateContentConfig(\n",
        "        system_instruction=system_instruction,\n",
        "        temperature=0.5,\n",
        "    ),\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JQem1halYDBW"
      },
      "outputs": [],
      "source": [
        "response = chat.send_message(\"Write a function that checks if a year is a leap year.\")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6Fn69TurZ9DB"
      },
      "outputs": [],
      "source": [
        "response = chat.send_message(\"Okay, write a unit test of the generated function.\")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rVlo0mWuZGkQ"
      },
      "source": [
        "## Control generated output\n",
        "\n",
        "The [controlled generation](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output) capability in Gemini API allows you to constraint the model output to a structured format. You can provide the schemas as Pydantic Models or a JSON string."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OjSgf2cDN_bG"
      },
      "outputs": [],
      "source": [
        "from pydantic import BaseModel\n",
        "\n",
        "\n",
        "class Recipe(BaseModel):\n",
        "    name: str\n",
        "    description: str\n",
        "    ingredients: list[str]\n",
        "\n",
        "\n",
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=\"List a few popular cookie recipes and their ingredients.\",\n",
        "    config=GenerateContentConfig(\n",
        "        response_mime_type=\"application/json\",\n",
        "        response_schema=Recipe,\n",
        "    ),\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nKai5CP_PGQF"
      },
      "source": [
        "Optionally, you can parse the response string to JSON."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZeyDWbnxO-on"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "json_response = json.loads(response.text)\n",
        "print(json.dumps(json_response, indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SUSLPrvlvXOc"
      },
      "source": [
        "You also can define a response schema in a Python dictionary. You can only use the supported fields as listed below. All other fields are ignored.\n",
        "\n",
        "- `enum`\n",
        "- `items`\n",
        "- `maxItems`\n",
        "- `nullable`\n",
        "- `properties`\n",
        "- `required`\n",
        "\n",
        "In this example, you instruct the model to analyze product review data, extract key entities, perform sentiment classification (multiple choices), provide additional explanation, and output the results in JSON format.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "F7duWOq3vMmS"
      },
      "outputs": [],
      "source": [
        "response_schema = {\n",
        "    \"type\": \"ARRAY\",\n",
        "    \"items\": {\n",
        "        \"type\": \"ARRAY\",\n",
        "        \"items\": {\n",
        "            \"type\": \"OBJECT\",\n",
        "            \"properties\": {\n",
        "                \"rating\": {\"type\": \"INTEGER\"},\n",
        "                \"flavor\": {\"type\": \"STRING\"},\n",
        "                \"sentiment\": {\n",
        "                    \"type\": \"STRING\",\n",
        "                    \"enum\": [\"POSITIVE\", \"NEGATIVE\", \"NEUTRAL\"],\n",
        "                },\n",
        "                \"explanation\": {\"type\": \"STRING\"},\n",
        "            },\n",
        "            \"required\": [\"rating\", \"flavor\", \"sentiment\", \"explanation\"],\n",
        "        },\n",
        "    },\n",
        "}\n",
        "\n",
        "prompt = \"\"\"\n",
        "  Analyze the following product reviews, output the sentiment classification and give an explanation.\n",
        "\n",
        "  - \"Absolutely loved it! Best ice cream I've ever had.\" Rating: 4, Flavor: Strawberry Cheesecake\n",
        "  - \"Quite good, but a bit too sweet for my taste.\" Rating: 1, Flavor: Mango Tango\n",
        "\"\"\"\n",
        "\n",
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=prompt,\n",
        "    config=GenerateContentConfig(\n",
        "        response_mime_type=\"application/json\",\n",
        "        response_schema=response_schema,\n",
        "    ),\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w9DRn59MZOoa"
      },
      "source": [
        "## Generate content stream\n",
        "\n",
        "By default, the model returns a response after completing the entire generation process. You can also use `generate_content_stream` method to stream the response as it is being generated, and the model will return chunks of the response as soon as they are generated."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ztOhpfznZSzo"
      },
      "outputs": [],
      "source": [
        "for chunk in client.models.generate_content_stream(\n",
        "    model=MODEL_ID,\n",
        "    contents=\"Tell me a story about a lonely robot who finds friendship in a most unexpected place.\",\n",
        "):\n",
        "    print(chunk.text)\n",
        "    print(\"*****************\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "arLJE4wOuhh6"
      },
      "source": [
        "## Send asynchronous requests\n",
        "\n",
        "You can send asynchronous requests using the `client.aio` module. This module exposes all the analogous async methods that are available on `client`.\n",
        "\n",
        "For example, `client.aio.models.generate_content` is the async version of `client.models.generate_content`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gSReaLazs-dP"
      },
      "outputs": [],
      "source": [
        "response = await client.aio.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=\"Compose a song about the adventures of a time-traveling squirrel.\",\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gV1dR-QlTKRs"
      },
      "source": [
        "## Count tokens and compute tokens\n",
        "\n",
        "You can use `count_tokens` method to calculates the number of input tokens before sending a request to the Gemini API. See the [List and count tokens](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/list-token) page for more details.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Syx-fwLkV1j-"
      },
      "source": [
        "#### Count tokens"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UhNElguLRRNK"
      },
      "outputs": [],
      "source": [
        "response = client.models.count_tokens(\n",
        "    model=MODEL_ID,\n",
        "    contents=\"What's the highest mountain in Africa?\",\n",
        ")\n",
        "\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VS-AP7AHUQmV"
      },
      "source": [
        "#### Compute tokens\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Cdhi5AX1TuH0"
      },
      "outputs": [],
      "source": [
        "response = client.models.compute_tokens(\n",
        "    model=MODEL_ID,\n",
        "    contents=\"What's the longest word in the English language?\",\n",
        ")\n",
        "\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T0pb-Kh1xEHU"
      },
      "source": [
        "## Function calling\n",
        "\n",
        "[Function calling](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling) lets you provide a set of tools that it can use to respond to the user's prompt. You create a description of a function in your code, then pass that description to a language model in a request. The response from the model includes the name of a function that matches the description and the arguments to call it with."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2BDQPwgcxRN3"
      },
      "outputs": [],
      "source": [
        "get_destination = FunctionDeclaration(\n",
        "    name=\"get_destination\",\n",
        "    description=\"Get the destination that the user wants to go to\",\n",
        "    parameters={\n",
        "        \"type\": \"OBJECT\",\n",
        "        \"properties\": {\n",
        "            \"destination\": {\n",
        "                \"type\": \"STRING\",\n",
        "                \"description\": \"Destination that the user wants to go to\",\n",
        "            },\n",
        "        },\n",
        "    },\n",
        ")\n",
        "\n",
        "destination_tool = Tool(\n",
        "    function_declarations=[get_destination],\n",
        ")\n",
        "\n",
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=\"I'd like to travel to Paris.\",\n",
        "    config=GenerateContentConfig(\n",
        "        tools=[destination_tool],\n",
        "        temperature=0,\n",
        "    ),\n",
        ")\n",
        "\n",
        "response.candidates[0].content.parts[0].function_call"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EA1Sn-VQE6_J"
      },
      "source": [
        "## Use context caching\n",
        "\n",
        "[Context caching](https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-overview) lets you to store frequently used input tokens in a dedicated cache and reference them for subsequent requests, eliminating the need to repeatedly pass the same set of tokens to a model.\n",
        "\n",
        "Context caching is only available for stable models with fixed versions (for example, `gemini-1.5-pro-002`). You must include the version postfix (for example, the `-002` in `gemini-1.5-pro-002`)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nqxTesUPIkNC"
      },
      "source": [
        "#### Create a cache"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "adsuvFDA6xP5"
      },
      "outputs": [],
      "source": [
        "system_instruction = \"\"\"\n",
        "  You are an expert researcher who has years of experience in conducting systematic literature surveys and meta-analyses of different topics.\n",
        "  You pride yourself on incredible accuracy and attention to detail. You always stick to the facts in the sources provided, and never make up new facts.\n",
        "  Now look at the research paper below, and answer the following questions in 1-2 sentences.\n",
        "\"\"\"\n",
        "\n",
        "pdf_parts = [\n",
        "    Part.from_uri(\n",
        "        \"gs://cloud-samples-data/generative-ai/pdf/2312.11805v3.pdf\",\n",
        "        mime_type=\"application/pdf\",\n",
        "    ),\n",
        "    Part.from_uri(\n",
        "        \"gs://cloud-samples-data/generative-ai/pdf/2403.05530.pdf\",\n",
        "        mime_type=\"application/pdf\",\n",
        "    ),\n",
        "]\n",
        "\n",
        "cached_content = client.caches.create(\n",
        "    model=\"gemini-1.5-flash-002\",\n",
        "    config=CreateCachedContentConfig(\n",
        "        system_instruction=system_instruction,\n",
        "        contents=pdf_parts,\n",
        "        ttl=\"3600s\",\n",
        "    ),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JBdQNHEoJmC5"
      },
      "source": [
        "#### Use a cache"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "N8EhgCzlIoFI"
      },
      "outputs": [],
      "source": [
        "response = client.models.generate_content(\n",
        "    model=\"gemini-1.5-flash-002\",\n",
        "    contents=\"What is the research goal shared by these research papers?\",\n",
        "    config=GenerateContentConfig(\n",
        "        cached_content=cached_content.name,\n",
        "    ),\n",
        ")\n",
        "\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "azhqrdiCer19"
      },
      "source": [
        "#### Delete a cache"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rAUYcfOUdeoi"
      },
      "outputs": [],
      "source": [
        "client.caches.delete(name=cached_content.name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "43be33d2672b"
      },
      "source": [
        "## Batch prediction\n",
        "\n",
        "Different from getting online (synchronous) responses, where you are limited to one input request at a time, [batch predictions for the Gemini API in Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/batch-prediction-gemini) allow you to send a large number of requests to Gemini in a single batch request. Then, the model responses asynchronously populate to your storage output location in [Cloud Storage](https://cloud.google.com/storage/docs/introduction) or [BigQuery](https://cloud.google.com/bigquery/docs/storage_overview).\n",
        "\n",
        "Batch predictions are generally more efficient and cost-effective than online predictions when processing a large number of inputs that are not latency sensitive."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "adf948ae326b"
      },
      "source": [
        "### Prepare batch inputs\n",
        "\n",
        "The input for batch requests specifies the items to send to your model for prediction.\n",
        "\n",
        "Batch requests for Gemini accept BigQuery storage sources and Cloud Storage sources. You can learn more about the batch input formats in the [Batch text generation](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/batch-prediction-gemini#prepare_your_inputs) page.\n",
        "\n",
        "This tutorial uses Cloud Storage as an example. The requirements for Cloud Storage input are:\n",
        "\n",
        "- File format: [JSON Lines (JSONL)](https://jsonlines.org/)\n",
        "- Located in `us-central1`\n",
        "- Appropriate read permissions for the service account\n",
        "\n",
        "Each request that you send to a model can include parameters that control how the model generates a response. Learn more about Gemini parameters in the [Experiment with parameter values](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/adjust-parameter-values) page.\n",
        "\n",
        "This is one of the example requests in the input JSONL file `batch_requests_for_multimodal_input_2.jsonl`:\n",
        "\n",
        "```json\n",
        "{\"request\":{\"contents\": [{\"role\": \"user\", \"parts\": [{\"text\": \"List objects in this image.\"}, {\"file_data\": {\"file_uri\": \"gs://cloud-samples-data/generative-ai/image/office-desk.jpeg\", \"mime_type\": \"image/jpeg\"}}]}],\"generationConfig\":{\"temperature\": 0.4}}}\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "81b25154a51a"
      },
      "outputs": [],
      "source": [
        "INPUT_DATA = \"gs://cloud-samples-data/generative-ai/batch/batch_requests_for_multimodal_input_2.jsonl\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2031bb3f44c2"
      },
      "source": [
        "### Prepare batch output location\n",
        "\n",
        "When a batch prediction task completes, the output is stored in the location that you specified in your request.\n",
        "\n",
        "- The location is in the form of a Cloud Storage or BigQuery URI prefix, for example:\n",
        "`gs://path/to/output/data` or `bq://projectId.bqDatasetId`.\n",
        "\n",
        "- If not specified, `gs://STAGING_BUCKET/gen-ai-batch-prediction` will be used for Cloud Storage source and `bq://PROJECT_ID.gen_ai_batch_prediction.predictions_TIMESTAMP` will be used for BigQuery source.\n",
        "\n",
        "This tutorial uses a Cloud Storage bucket as an example for the output location.\n",
        "\n",
        "- You can specify the URI of your Cloud Storage bucket in `BUCKET_URI`, or\n",
        "- if it is not specified, a new Cloud Storage bucket in the form of `gs://PROJECT_ID-TIMESTAMP` will be created for you."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "fddd98cd84cd"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = \"[your-cloud-storage-bucket]\"  # @param {type:\"string\"}\n",
        "\n",
        "if BUCKET_URI == \"[your-cloud-storage-bucket]\":\n",
        "    TIMESTAMP = datetime.now().strftime(\"%Y%m%d%H%M%S\")\n",
        "    BUCKET_URI = f\"gs://{PROJECT_ID}-{TIMESTAMP}\"\n",
        "\n",
        "    ! gsutil mb -l {LOCATION} -p {PROJECT_ID} {BUCKET_URI}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d7da62c98880"
      },
      "source": [
        "### Send a batch prediction request\n",
        "\n",
        "To make a batch prediction request, you specify a source model ID, an input source and an output location where Vertex AI stores the batch prediction results.\n",
        "\n",
        "To learn more, see the [Batch prediction API](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/batch-prediction-api) page.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7ed3c2925663"
      },
      "outputs": [],
      "source": [
        "batch_job = client.batches.create(\n",
        "    model=\"gemini-1.5-flash-002\",\n",
        "    src=INPUT_DATA,\n",
        "    config=CreateBatchJobConfig(dest=BUCKET_URI),\n",
        ")\n",
        "batch_job.name"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f1bd49ff2c9e"
      },
      "source": [
        "Print out the job status and other properties. You can also check the status in the Cloud Console at https://console.cloud.google.com/vertex-ai/batch-predictions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ee2ec586e4f1"
      },
      "outputs": [],
      "source": [
        "client.batches.get(name=batch_job.name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "64eaf082ecb0"
      },
      "source": [
        "Optionally, you can list all the batch prediction jobs in the project."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "da8e9d43a89b"
      },
      "outputs": [],
      "source": [
        "for job in client.batches.list():\n",
        "    print(job.name, job.create_time, job.state)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0156eaf66675"
      },
      "source": [
        "### Retrieve batch prediction results\n",
        "\n",
        "When a batch prediction task is complete, the output of the prediction is stored in the location that you specified in your request. It is also available in `batch_job.dest.bigquery_uri` or `batch_job.dest.gcs_uri`.\n",
        "\n",
        "Example output:\n",
        "\n",
        "```json\n",
        "{\"status\": \"\", \"processed_time\": \"2024-11-13T14:04:28.376+00:00\", \"request\": {\"contents\": [{\"parts\": [{\"file_data\": null, \"text\": \"List objects in this image.\"}, {\"file_data\": {\"file_uri\": \"gs://cloud-samples-data/generative-ai/image/gardening-tools.jpeg\", \"mime_type\": \"image/jpeg\"}, \"text\": null}], \"role\": \"user\"}], \"generationConfig\": {\"temperature\": 0.4}}, \"response\": {\"candidates\": [{\"avgLogprobs\": -0.10394711927934126, \"content\": {\"parts\": [{\"text\": \"Here's a list of the objects in the image:\\n\\n* **Watering can:** A green plastic watering can with a white rose head.\\n* **Plant:** A small plant (possibly oregano) in a terracotta pot.\\n* **Terracotta pots:** Two terracotta pots, one containing the plant and another empty, stacked on top of each other.\\n* **Gardening gloves:** A pair of striped gardening gloves.\\n* **Gardening tools:** A small trowel and a hand cultivator (hoe).  Both are green with black handles.\"}], \"role\": \"model\"}, \"finishReason\": \"STOP\"}], \"modelVersion\": \"gemini-1.5-flash-002@default\", \"usageMetadata\": {\"candidatesTokenCount\": 110, \"promptTokenCount\": 264, \"totalTokenCount\": 374}}}\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "c2ce0968112c"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "\n",
        "# Load the JSONL file into a DataFrame\n",
        "df = pd.read_json(f\"{batch_job.dest.gcs_uri}/*/predictions.jsonl\", lines=True)\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f81ccNPjiVzH"
      },
      "source": [
        "## Get text embeddings\n",
        "\n",
        "You can get text embeddings for a snippet of text by using `embed_content` method. All models produce an output with 768 dimensions by default. However, some models give users the option to choose an output dimensionality between `1` and `768`. See [Vertex AI text embeddings API](https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-text-embeddings) for more details."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zGOCzT7y31rk"
      },
      "outputs": [],
      "source": [
        "TEXT_EMBEDDING_MODEL_ID = \"text-embedding-005\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "s94DkG5JewHJ"
      },
      "outputs": [],
      "source": [
        "response = client.models.embed_content(\n",
        "    model=TEXT_EMBEDDING_MODEL_ID,\n",
        "    contents=[\n",
        "        \"How do I get a driver's license/learner's permit?\",\n",
        "        \"How do I renew my driver's license?\",\n",
        "        \"How do I change my address on my driver's license?\",\n",
        "    ],\n",
        "    config=EmbedContentConfig(output_dimensionality=128),\n",
        ")\n",
        "\n",
        "print(response.embeddings)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eQwiONFdVHw5"
      },
      "source": [
        "# What's next\n",
        "\n",
        "- Explore other notebooks in the [Google Cloud Generative AI GitHub repository](https://github.com/GoogleCloudPlatform/generative-ai).\n",
        "- Explore AI models in [Model Garden](https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models)."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "intro_genai_sdk.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}