Pandas Agent unable to use the trained data.

[Subhashchandra-P]

System Info

OS Version : windows 11
Python Version : 3.11.9
Pandas ai version : 2.2.14

🐛 Describe the bug

Thanks for the Great work!
As a part of training the agent (QA Training) ,started training on some questions...the training has successfully completed...but when try to run the trained question, it is unable to use that trianed data? why?what's the reason?

2024-11-08 06:13:03 [INFO] Running PandasAI with langchain_chat-databricks LLM...
2024-11-08 06:13:03 [INFO] Prompt ID: 413fcc0b-0d0f-4b4e-ad6e-d6db1be57cd2
2024-11-08 06:13:03 [INFO] Executing Pipeline: GenerateChatPipeline
2024-11-08 06:13:03 [INFO] Executing Step 0: ValidatePipelineInput
2024-11-08 06:13:03 [INFO] Executing Step 1: CacheLookup
2024-11-08 06:13:03 [INFO] Executing Step 2: PromptGeneration
2024-11-08 06:13:04 [INFO] Querying without using training data.
2024-11-08 06:13:04 [INFO] Querying without using training docs.
2024-11-08 06:13:04 [INFO] Using prompt:
@dosubot

[dosubot]

To address the issue of PandasAI not using the trained data, consider the following steps:

1. API Key Configuration: Ensure that your PandasAI API key is correctly set as an environment variable. This is crucial for accessing the BambooVectorStore where the training data is stored. Without this, the trained data cannot be utilized ^[1].

   import os
   os.environ["PANDASAI_API_KEY"] = "YOUR_PANDASAI_API_KEY"

2. Verbose Logging: Enable verbose logging to get more detailed information about the execution process. This can help identify why the trained data is not being used.

   smart_datalake_instance.verbose = True

3. Check Logs and Errors: Use the logs and last_error properties to check for any errors or issues that might indicate why the trained data is not being utilized.

   logs = smart_datalake_instance.logs
   last_error = smart_datalake_instance.last_error

4. Vector Store Configuration: Verify the configuration of your vector store. Ensure that the max_samples and similarity_threshold settings are appropriate for your use case. These settings affect how the trained data is retrieved and used during query execution ^[2].

If these steps do not resolve the issue, there might be specific conditions or configurations in the codebase that determine when trained data is bypassed, which are not explicitly documented.

To continue talking to Dosu, mention @dosu.

[Subhashchandra-P]

Instantiate the vector store with the API keys

vector_store = BambooVectorStor(api_key="YOUR_PANDASAI_API_KEY")

Instantiate the agent with the custom vector store

agent = Agent(connector, config={...} vectorstore=vector_store)

is this correct @dosu

[dosubot]

The instantiation looks mostly correct, but ensure the following:

1. Correct Class Name: Double-check the class name for the vector store. It should be BambooVectorStore instead of BambooVectorStor.

2. API Key: Make sure the API key is valid and correctly set.

3. Configuration: Ensure that the configuration dictionary (config={...}) is properly defined with all necessary parameters for your use case.

Here's a corrected version:

# Instantiate the vector store with the API key
vector_store = BambooVectorStore(api_key="YOUR_PANDASAI_API_KEY")

# Instantiate the agent with the custom vector store
agent = Agent(connector, config={...}, vectorstore=vector_store)

If you continue to experience issues, verify that the connector and config are correctly set up and compatible with your version of PandasAI.

To continue talking to Dosu, mention @dosu.

[Subhashchandra-P]

From where do we need to import BambooVectorStore?

vector_store = BambooVectorStore(api_key="YOUR_PANDASAI_API_KEY")
@dosu

[Subhashchandra-P]

From which library do we need to import BambooVectorStore?

vector_store = BambooVectorStore(api_key="YOUR_PANDASAI_API_KEY")
@dosu

[dosubot]

You can import BambooVectorStore from the pandasai.ee.vectorstores module. Here's how you can do it:

from pandasai.ee.vectorstores import BambooVectorStore

# Instantiate the vector store with the API key
vector_store = BambooVectorStore(api_key="YOUR_PANDASAI_API_KEY")

Make sure you have the necessary dependencies installed and that your environment is correctly set up to use this module.

To continue talking to Dosu, mention @dosu.