Theory on how to alleviate the token max issue.

[xbwtyz]

I see you are already working on a method using viewport to cutdown on token amount, but what if apply the following as well

• Take the simplified DOM and cut it into segments that fit
• Send it to the LLM with a modified version of the user's instructions such as "what portions of the following message are relevant to [user's instructions]: " etc.,
• store the return in a "master message"
• repeat steps 2-3 until all parts of the DOM are sent
• send the master message alongside the user's instructions so that only relevant portions of the DOM are sent.

This would of course introduce unnecessary space for error and ultimately slow down the extension, but maybe it would function in niche cases?

[CryptoMitch]

`async function getRelevantDomSegments(dom, instructions, llm) {
const segmentSize = 50; // or any other reasonable size
let masterMessage = "";
let startIndex = 0;
let endIndex = segmentSize;

while (startIndex < dom.length) {
const segment = dom.slice(startIndex, endIndex);
const modifiedInstructions = What portions of the following message are relevant to ${instructions}: ${segment};

const relevantPortion = await llm(modifiedInstructions); // assuming llm is an async function that sends the text to LLM and returns the result
masterMessage += relevantPortion;

startIndex += segmentSize;
endIndex += segmentSize;

}

const finalInstructions = ${instructions} ${masterMessage};
const result = await llm(finalInstructions);

return result;
}`

[ibrahimsk1]

@xbwtyz cool solution!

My suggestion is after simplifying the html creating indexes and keeping these. Afterwards, asking questions about the user interactions and getting the relavent parts.

[tluyben]

It seems that keeping this in html format to send to gpt is taking up a lot of space (tokens)? Wouldn't it be better to just generate a flat list of [id, type, text] and send that over? So like [1833, 'c' 'Read More'] where you can have types like 'c' for clickable, 'i' for inputable etc.

Edit; doing some testing here https://github.com/tluyben/browser-extension/tree/feature/token-limit-attempt ; seems to work better for sites i have tested.