Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeouts and VM exit for index building process. #8504

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

isoos
Copy link
Collaborator

@isoos isoos commented Jan 28, 2025

  • The callback will start a timer that, if not cancelled, will call exit to make sure the VM is restarted.
  • Added regular timeouts to storage download and also upload calls.

@isoos isoos requested review from jonasfj and sigurdm January 28, 2025 15:21
stderr.writeln('Timeout triggering VM termination\n$trace');
_logger.pubNoticeShout(
'vm-termination',
'Timeout triggering VM termination',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this terminate the VM or does AppEngine just restart the process?

await for (final package in dbService.query<Package>().run()) {
if (package.isNotVisible) {
continue;
await withVmTerminationTimeout(timeout: Duration(hours: 12), () async {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just add .timeout(Duration(hours: 12)) when calling doCreateAndUpdateSnapshot?

If such a timeout does happen there, isn't that a good place to just not catch the timeout and let it bubble all the way up? Do we need to explicitly kill the VM?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants