Ollama Experimenting

[dd36]

Are you getting the "AttributeError: 'str' object has no attribute 'get'" error? You're probably losing context.

I have been doing some experimenting with the default "Top 3 results on Hacker News" topic to figure out what is going on. My Top 3 results were always the bottom results from the HN list, which didn't make sense unless the top of the prompt was being lost.

This led me to discover that Ollama has a default context length of 2048. So if you just use a model without updating that context length, you'll get weird results because it only gets the end of the context. Skyvern passes the DOM/html, which can mean very large context, much larger than 2048. For example, many of the instructions, like outputting valid JSON, are at the beginning of the message but the beginning of the message gets lost to the context window.

The solution is to add a context size model parameter called num_ctx. The larger the number, the more memory your system needs to have. I could not get the full context processed even at num_ctx 8192. It worked with num_ctx 16384 but I did not try anything less as I have 128gb of RAM.

The simplest way to set this is by updating the Ollama model file. You can read how to do so here: https://help.nurgo-software.com/article/202-optimizing-ollama-models-for-brainsoup

I named my updated model as qwen3:30b-a3b-max_context. You then need to add this model to your docker-compose file.

So, I am now getting valid JSON. However, it is not giving me the top 3 results. Almost like it is taking the request as a request on how to get the Top 3 rather than a request to provide the Top 3. I will try to figure out why next. It may also be context related.

I am attaching my docker-compose.yml, request.json, and response.json.

[dd36]

2025-05-07T16:20:28.966533_a_390634270895595536_llm_request.json
2025-05-07T16:21:51.791245_a_390634627377881108_llm_response_parsed.json
docker-compose.txt

[dd36]

qwen3:14b with 16K context window worked on a taskv2:

[dd36]

It did not work to get Top 3.

[dd36]

Qwen3:14b with 40960 size context window did work to get Top 3.

[dd36]

It may not be on Skyvern's roadmap but optimizing an open source model and the prompts for that file may be a winning path.

[dd36]

qwen2.5vl:32b with an updated context window of 125k has my workflows working. It fails with the default context window.

7b failed with 125k and default. It seemed to have the right responses so I am wondering if there is some other issue.

[dd36]

Problem for most people will be 32b with 125k context window needs 51GB RAM. To do most of these tasks shouldn't need anywhere near that.

[The-BlackSmith872]

Hi there, I'm running a sample workflow using qwen2.5vl:32b with a modified context window of 125k, but as soon as it navigates to the page, it freezes as if the workflow has stopped.
May I know how long it took your workflow to return the top 3 results on Hackernews?