Skip to content

Commit 4230709

Browse files
ravi03071991ravi-theja
ravi03071991
and
ravi-theja
authored
Add google colab support for notebooks (run-llama#7560)
* Add google colab support for notebooks * Add google colab support for CustomRetriever Notebook. * Add google colab support for query_engine Notebooks. * Add google colab support for response_synthesizers Notebooks. * Add google colab support for query_transformations Notebooks. * Add google colab support for retrievers Notebooks. * Add google colab support for managed Notebooks. * Add google colab support for callbacks Notebooks. * Add google colab support for chat engine Notebooks. * Add google colab support for citation Notebooks. * Add google colab support for composable indices Notebooks. * Fix formatting issues * Add google colab support for llm Notebooks. * Add google colab support for customization Notebooks. * Add google colab support for data connectors" Notebooks. * Add google colab support for docstore Notebooks. * Fix formatting issues and Add google colab support for embeddings Notebooks. * Fix format issues * Add google colab support for Evaluation Notebooks * Add google colab support for finetuning Notebooks * Add google colab support for low level Notebooks * Add google colab support for index_structs notebooks * Add google colab support for llm notebooks * Add google colab support for metadata extraction notebooks * Add google colab support for tools notebooks * Fix linting issues * Add google colab support for vector store notebooks. * Add google colab support for usecases * Add google colab support for output parsing notebooks * Add google colab notebook support for node postprocessors notebook * Add google colab support for agent and anlysis notebooks * Update download data code * Update data links * Add google colab support * Add google colab support * Add google colab support for notebooks --------- Co-authored-by: Ravi Theja <[email protected]>
1 parent 5717dd8 commit 4230709

File tree

253 files changed

+8279
-348
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

253 files changed

+8279
-348
lines changed

‎.pre-commit-config.yaml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,19 +16,19 @@ repos:
1616
- id: mixed-line-ending
1717
- id: trailing-whitespace
1818
- repo: https://github.com/charliermarsh/ruff-pre-commit
19-
rev: v0.1.0
19+
rev: v0.1.2
2020

2121
hooks:
2222
- id: ruff
2323
args: [--fix, --exit-non-zero-on-fix]
2424
- repo: https://github.com/psf/black-pre-commit-mirror
25-
rev: 23.10.0
25+
rev: 23.10.1
2626
hooks:
2727
- id: black-jupyter
2828
name: black-src
2929
exclude: docs/
3030
- repo: https://github.com/psf/black-pre-commit-mirror
31-
rev: 23.10.0
31+
rev: 23.10.1
3232
hooks:
3333
- id: black-jupyter
3434
name: black-docs
@@ -44,7 +44,7 @@ repos:
4444
- id: codespell
4545
additional_dependencies: [tomli]
4646
- repo: https://github.com/srstevenson/nb-clean
47-
rev: 3.0.1
47+
rev: 3.1.0
4848
hooks:
4949
- id: nb-clean
5050
args: [--preserve-cell-outputs, --remove-empty-cells]

‎docs/examples/agent/Chatbot_SEC.ipynb‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
{
22
"cells": [
3+
{
4+
"attachments": {},
5+
"cell_type": "markdown",
6+
"id": "e8c3c7da",
7+
"metadata": {},
8+
"source": [
9+
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/agent/Chatbot_SEC.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
10+
]
11+
},
312
{
413
"attachments": {},
514
"cell_type": "markdown",

‎docs/examples/agent/multi_document_agents-v1.ipynb‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
{
22
"cells": [
3+
{
4+
"attachments": {},
5+
"cell_type": "markdown",
6+
"id": "91c998a5",
7+
"metadata": {},
8+
"source": [
9+
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/agent/multi_document_agents-v1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
10+
]
11+
},
312
{
413
"cell_type": "markdown",
514
"id": "43497beb-817d-4366-9156-f4d7f0d44942",
@@ -20,6 +29,25 @@
2029
"- setup a top-level agent over this set of document agents. Do tool retrieval and then do CoT over the set of tools to answer a question."
2130
]
2231
},
32+
{
33+
"attachments": {},
34+
"cell_type": "markdown",
35+
"id": "77ac7184",
36+
"metadata": {},
37+
"source": [
38+
"If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
39+
]
40+
},
41+
{
42+
"cell_type": "code",
43+
"execution_count": null,
44+
"id": "4eff88ab",
45+
"metadata": {},
46+
"outputs": [],
47+
"source": [
48+
"!pip install llama-index"
49+
]
50+
},
2351
{
2452
"cell_type": "code",
2553
"execution_count": null,

‎docs/examples/agent/multi_document_agents.ipynb‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
{
22
"cells": [
3+
{
4+
"attachments": {},
5+
"cell_type": "markdown",
6+
"id": "67c8afd7",
7+
"metadata": {},
8+
"source": [
9+
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/agent/multi_document_agents.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
10+
]
11+
},
312
{
413
"cell_type": "markdown",
514
"id": "43497beb-817d-4366-9156-f4d7f0d44942",
@@ -34,6 +43,25 @@
3443
"We load in 18 cities - this is not quite at the level of \"hundreds\" of documents but its still large enough to warrant some top-level document retrieval!"
3544
]
3645
},
46+
{
47+
"attachments": {},
48+
"cell_type": "markdown",
49+
"id": "5d81f93c",
50+
"metadata": {},
51+
"source": [
52+
"If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
53+
]
54+
},
55+
{
56+
"cell_type": "code",
57+
"execution_count": null,
58+
"id": "f0fb1340",
59+
"metadata": {},
60+
"outputs": [],
61+
"source": [
62+
"!pip install llama-index"
63+
]
64+
},
3765
{
3866
"cell_type": "code",
3967
"execution_count": null,

‎docs/examples/agent/openai_agent.ipynb‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
{
22
"cells": [
3+
{
4+
"attachments": {},
5+
"cell_type": "markdown",
6+
"id": "24103c51",
7+
"metadata": {},
8+
"source": [
9+
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/agent/openai_agent.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
10+
]
11+
},
312
{
413
"cell_type": "markdown",
514
"id": "99cea58c-48bc-4af6-8358-df9695659983",
@@ -39,6 +48,25 @@
3948
"3. a definition for tools that our agent can use."
4049
]
4150
},
51+
{
52+
"attachments": {},
53+
"cell_type": "markdown",
54+
"id": "41101795",
55+
"metadata": {},
56+
"source": [
57+
"If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.\n"
58+
]
59+
},
60+
{
61+
"cell_type": "code",
62+
"execution_count": null,
63+
"id": "c61c873d",
64+
"metadata": {},
65+
"outputs": [],
66+
"source": [
67+
"!pip install llama-index"
68+
]
69+
},
4270
{
4371
"cell_type": "code",
4472
"execution_count": null,

‎docs/examples/agent/openai_agent_context_retrieval.ipynb‎

Lines changed: 53 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
{
22
"cells": [
3+
{
4+
"attachments": {},
5+
"cell_type": "markdown",
6+
"id": "9e34586b",
7+
"metadata": {},
8+
"source": [
9+
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/agent/openai_agent_context_retrieval.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
10+
]
11+
},
312
{
413
"attachments": {},
514
"cell_type": "markdown",
@@ -37,6 +46,25 @@
3746
"Here we setup a ContextRetrieverOpenAIAgent. This agent will perform retrieval first before calling any tools. This can help ground the agent's tool picking and answering capabilities in context."
3847
]
3948
},
49+
{
50+
"attachments": {},
51+
"cell_type": "markdown",
52+
"id": "d6d2e0ae",
53+
"metadata": {},
54+
"source": [
55+
"If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
56+
]
57+
},
58+
{
59+
"cell_type": "code",
60+
"execution_count": null,
61+
"id": "21770827",
62+
"metadata": {},
63+
"outputs": [],
64+
"source": [
65+
"!pip install llama-index"
66+
]
67+
},
4068
{
4169
"cell_type": "code",
4270
"execution_count": null,
@@ -84,6 +112,28 @@
84112
" index_loaded = False"
85113
]
86114
},
115+
{
116+
"attachments": {},
117+
"cell_type": "markdown",
118+
"id": "f9010d88",
119+
"metadata": {},
120+
"source": [
121+
"Download Data"
122+
]
123+
},
124+
{
125+
"cell_type": "code",
126+
"execution_count": null,
127+
"id": "75aa7b79",
128+
"metadata": {},
129+
"outputs": [],
130+
"source": [
131+
"!mkdir -p 'data/10q/'\n",
132+
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/10q/uber_10q_march_2022.pdf' -O 'data/10q/uber_10q_march_2022.pdf'\n",
133+
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/10q/uber_10q_june_2022.pdf' -O 'data/10q/uber_10q_june_2022.pdf'\n",
134+
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/10q/uber_10q_sept_2022.pdf' -O 'data/10q/uber_10q_sept_2022.pdf'"
135+
]
136+
},
87137
{
88138
"cell_type": "code",
89139
"execution_count": null,
@@ -96,13 +146,13 @@
96146
"if not index_loaded:\n",
97147
" # load data\n",
98148
" march_docs = SimpleDirectoryReader(\n",
99-
" input_files=[\"../data/10q/uber_10q_march_2022.pdf\"]\n",
149+
" input_files=[\"./data/10q/uber_10q_march_2022.pdf\"]\n",
100150
" ).load_data()\n",
101151
" june_docs = SimpleDirectoryReader(\n",
102-
" input_files=[\"../data/10q/uber_10q_june_2022.pdf\"]\n",
152+
" input_files=[\"./data/10q/uber_10q_june_2022.pdf\"]\n",
103153
" ).load_data()\n",
104154
" sept_docs = SimpleDirectoryReader(\n",
105-
" input_files=[\"../data/10q/uber_10q_sept_2022.pdf\"]\n",
155+
" input_files=[\"./data/10q/uber_10q_sept_2022.pdf\"]\n",
106156
" ).load_data()\n",
107157
"\n",
108158
" # build index\n",

‎docs/examples/agent/openai_agent_query_cookbook.ipynb‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
{
22
"cells": [
3+
{
4+
"attachments": {},
5+
"cell_type": "markdown",
6+
"id": "70b86d36",
7+
"metadata": {},
8+
"source": [
9+
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/agent/openai_agent_query_cookbook.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
10+
]
11+
},
312
{
413
"attachments": {},
514
"cell_type": "markdown",
@@ -28,6 +37,25 @@
2837
"Since the OpenAI Function API can infer function parameters, we explore its capabilities in performing auto-retrieval here."
2938
]
3039
},
40+
{
41+
"attachments": {},
42+
"cell_type": "markdown",
43+
"id": "50444bd8",
44+
"metadata": {},
45+
"source": [
46+
"If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
47+
]
48+
},
49+
{
50+
"cell_type": "code",
51+
"execution_count": null,
52+
"id": "901d9a0c",
53+
"metadata": {},
54+
"outputs": [],
55+
"source": [
56+
"!pip install llama-index"
57+
]
58+
},
3159
{
3260
"cell_type": "code",
3361
"execution_count": null,

‎docs/examples/agent/openai_agent_query_plan.ipynb‎

Lines changed: 53 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
{
22
"cells": [
3+
{
4+
"attachments": {},
5+
"cell_type": "markdown",
6+
"id": "fa593487",
7+
"metadata": {},
8+
"source": [
9+
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/agent/openai_agent_query_plan.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
10+
]
11+
},
312
{
413
"attachments": {},
514
"cell_type": "markdown",
@@ -17,6 +26,25 @@
1726
"In this setting we use a familiar example: Uber 10Q filings in March, June, and September of 2022."
1827
]
1928
},
29+
{
30+
"attachments": {},
31+
"cell_type": "markdown",
32+
"id": "dc72e6f9",
33+
"metadata": {},
34+
"source": [
35+
"If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙."
36+
]
37+
},
38+
{
39+
"cell_type": "code",
40+
"execution_count": null,
41+
"id": "4df080d3",
42+
"metadata": {},
43+
"outputs": [],
44+
"source": [
45+
"!pip install llama-index"
46+
]
47+
},
2048
{
2149
"cell_type": "code",
2250
"execution_count": null,
@@ -71,6 +99,28 @@
7199
"service_context = ServiceContext.from_defaults(llm=llm)"
72100
]
73101
},
102+
{
103+
"attachments": {},
104+
"cell_type": "markdown",
105+
"id": "26f545cd",
106+
"metadata": {},
107+
"source": [
108+
"## Download Data"
109+
]
110+
},
111+
{
112+
"cell_type": "code",
113+
"execution_count": null,
114+
"id": "e6385d12",
115+
"metadata": {},
116+
"outputs": [],
117+
"source": [
118+
"!mkdir -p 'data/10q/'\n",
119+
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/10q/uber_10q_march_2022.pdf' -O 'data/10q/uber_10q_march_2022.pdf'\n",
120+
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/10q/uber_10q_june_2022.pdf' -O 'data/10q/uber_10q_june_2022.pdf'\n",
121+
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/10q/uber_10q_sept_2022.pdf' -O 'data/10q/uber_10q_sept_2022.pdf'"
122+
]
123+
},
74124
{
75125
"attachments": {},
76126
"cell_type": "markdown",
@@ -88,13 +138,13 @@
88138
"outputs": [],
89139
"source": [
90140
"march_2022 = SimpleDirectoryReader(\n",
91-
" input_files=[\"../data/10q/uber_10q_march_2022.pdf\"]\n",
141+
" input_files=[\"./data/10q/uber_10q_march_2022.pdf\"]\n",
92142
").load_data()\n",
93143
"june_2022 = SimpleDirectoryReader(\n",
94-
" input_files=[\"../data/10q/uber_10q_june_2022.pdf\"]\n",
144+
" input_files=[\"./data/10q/uber_10q_june_2022.pdf\"]\n",
95145
").load_data()\n",
96146
"sept_2022 = SimpleDirectoryReader(\n",
97-
" input_files=[\"../data/10q/uber_10q_sept_2022.pdf\"]\n",
147+
" input_files=[\"./data/10q/uber_10q_sept_2022.pdf\"]\n",
98148
").load_data()"
99149
]
100150
},

0 commit comments

Comments
 (0)