Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pathogen-repo-build: Generate a Snakemake report for workflow #100

Open
joverlee521 opened this issue Jun 18, 2024 · 6 comments
Open

pathogen-repo-build: Generate a Snakemake report for workflow #100

joverlee521 opened this issue Jun 18, 2024 · 6 comments
Labels
enhancement New feature or request

Comments

@joverlee521
Copy link
Contributor

Context

Originally proposed in nextstrain/cli#373 (comment)

It'd be helpful to have a summary of workflow stats in the form of a Snakemake report for our automated pathogen workflows.

@joverlee521 joverlee521 added the enhancement New feature or request label Jun 18, 2024
@joverlee521
Copy link
Contributor Author

At first I thought this would be pretty straight-forward:

  1. Add another step to the GH Action workflow to run nextstrain build . --report
  2. Add report.html to the list of paths for the build artifact

However, the command nextstrain build . --report will not always work since the run input allows any arbitrary shell command, e.g. if the run command is nextstrain build ingest ..., we would need to run nextstrain build ingest --report.

@joverlee521
Copy link
Contributor Author

For non-aws-batch runtimes, I think this can be achieved directly through the run input

run: nextstrain build ingest; nextstrain build ingest --report

@joverlee521
Copy link
Contributor Author

For non-aws-batch runtimes, I think this can be achieved directly through the run input

run: nextstrain build ingest; nextstrain build ingest --report

Yup, this totally worked in zika/ingest (wip, artifact)

@joverlee521
Copy link
Contributor Author

We cannot follow the same pattern in the aws-batch runtime because of the wait for the AWS Batch job to finish...

Ah, we can download the Snakemake files into the current dir then generate the report (dependent on release with nextstrain/cli#374):

nextstrain build --aws-batch --attach <job-id> --download ".snakemake/" . ;
nextstrain build . --report

I think this does not have to be built into the pathogen-repo-build workflow as it can just be run as a separate job in the caller workflow (small block on #79)

I guess we can see how helpful Snakemake reports are for now with these alternative methods of generating them before we add more complexity to the pathogen-repo-build workflow.

@tsibley
Copy link
Member

tsibley commented Jun 18, 2024

(dependent on release with nextstrain/cli#374):

@joverlee521 Want to release 8.5.0 of Nextstrain CLI?

@joverlee521
Copy link
Contributor Author

(dependent on release with nextstrain/cli#374):

@joverlee521 Want to release 8.5.0 of Nextstrain CLI?

Yup! Will do ~now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants