bug: Image Modality is not working as expected.

[shyam1819]

Did you check docs and existing issues?

• I have read all the NeMo-Guardrails docs
• I have updated the package to the latest version before submitting this issue
• (optional) I have used the develop branch
• I have searched the existing issues of NeMo-Guardrails

Python version (python --version)

3.12

Operating system/version

Windows

NeMo-Guardrails version (if you must use a specific version and not the latest

0.20.0

Describe the bug

The base64 string is going as part of content and the request is failing due to the context size. Where as the image is well within the limits.

Steps To Reproduce

To reproduce I followed the steps provided in the multimodal docs of Nemo Guardrails.

Expected Behavior

Actually the content object itself is being passed as a string to the model.

It should be like this

Actual Behavior

As mentioned above

[shyam1819]

@Pouyanpi can you explain why the image base64 string is being sent as it is.

[shyam1819]

The action itself is receiving the user input as string.

[Pouyanpi]

Thank you @shyam1819 for reporting this bug.

I've opened ‪#1636 would you please use that branch and verify the issue is resolved?

[shyam1819]

Still facing the same issue.

I have read through the changes. I believe with these changes the images are not going to be moderated.

But I need image and text moderation capabilities.