pandas_dataframe_agent

[pgvetrivel]

Hi team,

i used create_pandas_dataframe_agent and GoogleGenerativeAI to create a chat. My dataframe has around 3000 rows. But, it seems like the model only access only the part of the dataframe and answers the query based on that. is there any limitation with the model? or I am making any mistake in coding?

import vertexai
from langchain_google_vertexai import ChatVertexAI
from langchain_experimental.agents import create_pandas_dataframe_agent
from langchain.agents.agent_types import AgentType
from langchain_google_genai import GoogleGenerativeAI
import pandas as pd

df = pd.read_csv('D:/Dialogflow/netsales_oct2024.csv')
llm = GoogleGenerativeAI(model="gemini-1.5-pro-002")

#Creating System Instruction
prompt = """You are a helpful agent providing netsales information for our partners (MFDs/IFAs). You can identify partners by their name or ARN code.

Your task is to construct a concise and informative sentence that directly answers a given query, incorporating the numerical response generated by another model.

Contextual Information:
* ARN, MFD, and IFA are interchangeable terms referring to partners.
* `txnmonth` represents the transaction month and follows the format 'YYYY-MM'.
* 'cr' stands for crores.
* 'lac' and 'lakh' are interchangeable.
* The financial year starts from April.
\""""

# Create a model instance
agent = create_pandas_dataframe_agent(llm, df, verbose=False, allow_dangerous_code=True)

messages = []
while True:

    human_message = input("You: ")
    if human_message.lower() == "quit":
        break
    messages.append({"role": "human", "content": human_message})
       
    response = agent.invoke(prompt + human_message)
    model_response = response['output']
    messages.append({"role": "ai", "content": model_response})
    print(f"AI: {model_response}")    

print("Conversation ended.")

[feijoes]

@pgvetrivel set number_of_head_rows=max_rows_df_arrys, but that increase the number of prompt tokens
docs

[pgvetrivel]

Thank you