Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable multithreading to avoid 6hr time-out during github action #45

Merged
merged 1 commit into from
Nov 20, 2024
Merged

Enable multithreading to avoid 6hr time-out during github action #45

merged 1 commit into from
Nov 20, 2024

Conversation

j23414
Copy link
Collaborator

@j23414 j23414 commented Nov 19, 2024 •
edited
Loading

Description of proposed changes

Add multithreading with automatic detection of threads number. Mainly addressing the issue below

Related issue(s)

Checklist

  • Checks pass

@j23414 j23414 linked an issue Nov 19, 2024 that may be closed by this pull request
Ingest to phylogenetic workflow fails because job exceeds 6hrs #44
Closed
joverlee521
joverlee521 approved these changes Nov 19, 2024
View reviewed changes
Copy link
Contributor

@joverlee521 joverlee521 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes @j23414!

ingest/profiles/default/config.yaml
Comment on lines +1 to +2
cores: all
printshellcmds: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not asking for a change

In case it's not clear, these configs are not necessary when running the builds with the Nextstrain CLI via nextstrain build.

  1. CLI will use all cores if the --cpus option is not provided (ref).
  2. The --printshellcmds flag is passed to snakemake by default (ref)

@j23414
Copy link
Collaborator Author

j23414 commented Nov 20, 2024

Below 6hrs: https://github.com/nextstrain/WNV/actions/runs/11920429171

@j23414 j23414 merged commit 828e323 into main Nov 20, 2024
43 checks passed
@j23414 j23414 deleted the increase-threads-to-get-below-six-hours branch November 20, 2024 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@joverlee521 joverlee521 joverlee521 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Ingest to phylogenetic workflow fails because job exceeds 6hrs
2 participants
@j23414 @joverlee521