Exception: Prompt exceeds max length of 16000 characters even without history

[ayoubazaouyat]

python call :
from nemoguardrails import LLMRails

rails = LLMRails(config)

messages=[{
"role": "user", "content": "what is an mbr ?"
}]

options = {"output_vars": True}

output = rails.generate(messages=messages, options=options)
print(output)

config.yml :
models:

• type: main
  engine: azure
  model: gpt-4o
  parameters:
  deployment_name: ****
  api_version: 2023-09-01-preview

rails:
output:
flows:
- self check facts

prompts:

• task: self_check_facts
  max_length: 120000
  content: |-
  You are given a task to identify if the hypothesis is grounded and entailed to the evidence.
  You will only use the contents of the evidence and not rely on external knowledge.
  Answer with yes/no. "evidence": {{ evidence }} "hypothesis": {{ response }} "entails":

---

error : Error while execution 'self_check_facts' with parameters :{ ***************}
Prompt exceeds max length of 16000 characters even without history
Traceback (most recent call last):
File "c:\Users\Ayoub_Azaouyat\AppData\Local\Programs\Python\Python311\Lib\site-packages\nemoguardrails\actions\action_dispatcher.py", line 197, in execute_action
result = await result
^^^^^^^^^^^^
File "C:\Users\Ayoub_Azaouyat\AppData\Local\Programs\Python\Python311\Lib\site-packages\nemoguardrails\library\self_check\facts\actions.py", line 45, in self_check_facts
prompt = llm_task_manager.render_task_prompt(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\Users\Ayoub_Azaouyat\AppData\Local\Programs\Python\Python311\Lib\site-packages\nemoguardrails\llm\taskmanager.py", line 231, in render_task_prompt
raise Exception(
Exception: Prompt exceeds max length of 16000 characters even without history

[ayoubazaouyat]

even after sitting the max_token to a lower limit it still give back the 12000 token limit, I even did change the default on the rails config lib, but it still shows the same error

[Pouyanpi]

Hi @ayoubazaouyat , would you try it out by using fact_checking instead of self_check_facts. So just change the task name. I will investigate it further if it was not resolved. Thanks!

[ayoubazaouyat]

Hi @Pouyanpi , I changed the following :

---

config.yml :

models:

• type: main
  engine: azure
  model: gpt-4o
  parameters:
  deployment_name: gpt-4o
  api_version: 2023-09-01-preview

rails:
output:
flows:
- fact checking

---

prompt.yml:
prompts:

• task: fact_checking
  models: [main]
  max_length: 120000
  content: |-
  You are given a task to identify if the hypothesis is grounded and entailed to the evidence.
  You will only use the contents of the evidence and not rely on external knowledge.
  Answer with yes/no. "evidence": {{ evidence }} "hypothesis": {{ response }} "entails":

---

my output.co looks as follows :
define user ask questions
"Can I connect to scales ?"
"Can I automate decisions?"

define flow answer pas-x question
user ...
$answer = execute rag()
$check_facts = True
bot $answer

---

error :

---

ValueError Traceback (most recent call last)
Cell In[11], line 1
----> 1 rails = LLMRails(config)
3 messages=[{
4 "role": "user", "content": "what is an mbr ?"
5 }]
7 options = {"output_vars": True}

File c:\Users\Ayoub_Azaouyat\AppData\Local\Programs\Python\Python311\Lib\site-packages\nemoguardrails\rails\llm\llmrails.py:211, in LLMRails.init(self, config, llm, verbose)
208 break
210 # We run some additional checks on the config
--> 211 self._validate_config()
213 # Next, we initialize the LLM engines (main engine and action engines if specified).
214 self._init_llms()

File c:\Users\Ayoub_Azaouyat\AppData\Local\Programs\Python\Python311\Lib\site-packages\nemoguardrails\rails\llm\llmrails.py:272, in LLMRails._validate_config(self)
270 for flow_name in self.config.rails.output.flows:
271 if flow_name not in existing_flows_names:
--> 272 raise ValueError(
273 f"The provided output rail flow {flow_name} does not exist"
274 )
276 for flow_name in self.config.rails.retrieval.flows:
277 if flow_name not in existing_flows_names:

ValueError: The provided output rail flow fact checking does not exist

---

what I want to do is use the default fact-checker or do I have to use something else, how can I fix it ? thanks

[Pouyanpi]

@ayoubazaouyat, I meant just to change the prompt's task:

task: fact_checking
  max_length: 120000
  content: |-
  You are given a task to identify if the hypothesis is grounded and entailed to the evidence.
  You will only use the contents of the evidence and not rely on external knowledge.
  Answer with yes/no. "evidence": {{ evidence }} "hypothesis": {{ response }} "entails":

keep the rest as it was.

rails:
  output:
    flows:
      - self check facts

would you please check it with this config?

[ayoubazaouyat]

@Pouyanpi , i did try that, it give this error back :

{
"name": "ValidationError",
"message": "1 validation error for RailsConfig
Value error, You must provide a self_check_facts prompt template. [type=value_error, input_value={'user_messages': {'ask':...onfig_path': './config'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.7/v/value_error",
"stack": "---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
Cell In[11], line 3
1 from nemoguardrails import RailsConfig
----> 3 config = RailsConfig.from_path("./config")

File ~/Downloads/Python/.venv/lib/python3.10/site-packages/nemoguardrails/rails/llm/config.py:856, in RailsConfig.from_path(config_path)
852 raw_config["instructions"] = _default_config["instructions"]
854 raw_config["config_path"] = config_path
--> 856 return RailsConfig.parse_object(raw_config)

File ~/Downloads/Python/.venv/lib/python3.10/site-packages/nemoguardrails/rails/llm/config.py:926, in RailsConfig.parse_object(cls, obj)
921 if flow_data.get("elements") and not flow_data["elements"][0].get(
922 "_type"
923 ):
924 flow_data["elements"] = parse_flow_elements(flow_data["elements"])
--> 926 return RailsConfig.parse_obj(obj)

File ~/Downloads/Python/.venv/lib/python3.10/site-packages/pydantic/main.py:1118, in BaseModel.parse_obj(cls, obj)
1112 @classmethod
1113 @typing_extensions.deprecated('The parse_obj method is deprecated; use model_validate instead.', category=None)
1114 def parse_obj(cls: type[Model], obj: Any) -> Model: # noqa: D102
1115 warnings.warn(
1116 'The parse_obj method is deprecated; use model_validate instead.', category=PydanticDeprecatedSince20
1117 )
-> 1118 return cls.model_validate(obj)

File ~/Downloads/Python/.venv/lib/python3.10/site-packages/pydantic/main.py:551, in BaseModel.model_validate(cls, obj, strict, from_attributes, context)
549 # __tracebackhide__ tells pytest and some other tools to omit this function from tracebacks
550 tracebackhide = True
--> 551 return cls.pydantic_validator.validate_python(
552 obj, strict=strict, from_attributes=from_attributes, context=context
553 )

ValidationError: 1 validation error for RailsConfig
Value error, You must provide a self_check_facts prompt template. [type=value_error, input_value={'user_messages': {'ask':...onfig_path': './config'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.7/v/value_error"
}

[Pouyanpi]

Thank you very much @ayoubazaouyat. I'll investigate it and update you shortly.

[Pouyanpi]

@ayoubazaouyat there seems to be a bug, thanks for reporting it.

As a workaround, before we fix it, you can use following in your prompts.yml:

prompts:

  - task: self_check_facts
    max_length: 120000
    content: |-
      You are given a task to identify if the hypothesis is grounded and entailed to the evidence.
      You will only use the contents of the evidence and not rely on external knowledge.
      Answer with yes/no. "evidence": {{ evidence }} "hypothesis": {{ response }} "entails":

  - task: fact_checking
    max_length: 120000
    content: |-
      You are given a task to identify if the hypothesis is grounded and entailed to the evidence.
      You will only use the contents of the evidence and not rely on external knowledge.
      Answer with yes/no. "evidence": {{ evidence }} "hypothesis": {{ response }} "entails":

It worked on my end,I've used the head of develop branch. I hope it temporarily resolves your issue.