Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Updated ADAPTER_LLMW_MAX_POLLS to 120 for 1 hour extraction #904

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: Updated ADAPTER_LLMW_MAX_POLLS to 120 for 1 hour extraction #904

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions backend/sample.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,8 +120,8 @@ SESSION_COOKIE_AGE=86400
# Time in seconds to wait before polling LLMWhisperer's status API
ADAPTER_LLMW_POLL_INTERVAL=30
# Total number of times to poll the status API.
# 500 mins to allow 1500 (max pages limit) * 20 (approx time in sec to process a page)
ADAPTER_LLMW_MAX_POLLS=1000
# ~60 mins (assuming it'll be enough to process 1500 pages with LLMW v2)
ADAPTER_LLMW_MAX_POLLS=120
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chandrasekharan-zipstack but what about for the V1? Doesn't it use the same ENV?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ritwik-g you're right, v1 uses the same envs. Shall we discourage support for v1 by changing this env (reducing the time / max possible pages) and instead enforce support with v2 for more number of pages?
Realistically speaking - I doubt if any user has such large extraction times.

Worst case,

  1. either we'll have to let this env be and take action after we sunset v1
  2. or introduce a new set of envs for v2 and update that (involves changes in the SDK, so I'm not a fan of this)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chandrasekharan-zipstack Will this not cause timeout if it exceeds max 15min time which we have set in gunicorn ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harini-venkataraman the timeout exists only for extractions that happens with a web UI. This large setting is mainly ideal for async pipeline based extractions where such gunicorn timeouts will not play a role

# Number of times to retry the /whisper-status API before failing the extraction
ADAPTER_LLMW_STATUS_RETRIES=5

Expand Down
Loading