Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure that recent BioPerl easyconfigs use Bundle easyblock #21136

Merged
merged 1 commit into from
Aug 6, 2024

Conversation

ocaisa
Copy link
Member

@ocaisa ocaisa commented Aug 6, 2024

This was causing problems in EESSI/software-layer#658 as eb was attempting to copy the same easyblock twice and that was failing if EasyBuild had a read-only installation.

@ocaisa
Copy link
Member Author

ocaisa commented Aug 6, 2024

@boegelbot please test @ generoso

@boegelbot
Copy link
Collaborator

@ocaisa: Request for testing this PR well received on login1

PR test command 'EB_PR=21136 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_21136 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 14026

Test results coming soon (I hope)...

- notification for comment with ID 2271373429 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 3 out of 3 (3 easyconfigs in total)
cns3 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/3d2f3cdecc439fe25f766a37ea83c302 for a full test report.

@ocaisa
Copy link
Member Author

ocaisa commented Aug 6, 2024

@boegelbot please test @ jsc-zen3

@boegelbot
Copy link
Collaborator

@ocaisa: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=21136 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_21136 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 4646

Test results coming soon (I hope)...

- notification for comment with ID 2271415934 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 3 out of 3 (3 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/e3bc2263351e1f1caa39f14697b4734a for a full test report.

@boegel boegel added the bug fix label Aug 6, 2024
@boegel boegel added this to the release after 4.9.2 milestone Aug 6, 2024
@boegel boegel changed the title Ensure BioPerl easyconfigs use Bundle easyblock Ensure that recent BioPerl easyconfigs use Bundle easyblock Aug 6, 2024
Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel
Copy link
Member

boegel commented Aug 6, 2024

I guess the intention with using PerlModule rather than Bundle was to avoid that these easyconfigs need to set PERL5LIB via modextrapaths, but they're still doing it.

@ocaisa Can you clarify why using PerlModule rather than Bundle is actually causing trouble here?
What is Bundle doing differently that the "Permission denied" error that occurs when using PerlModule doesn't happen?

@ocaisa
Copy link
Member Author

ocaisa commented Aug 6, 2024

The top-level easyblock is copied over to the reprod directory and then the extensions are installed. The extensions also copy their easyblocks to the reprod directory, but they are using the same easyblock. This is fine when the permissions on the original file allow it to be overwritten, but when you start from a read-only installation of EasyBuild you end up with a permissions issue. If we switch to Bundle as the top-level easyblock we sidestep the problem.

The comment in easybuilders/easybuild-framework#4602 (comment) still stands, the framework could be a little more graceful here and recognise that the files have the same content (so the copy could be skipped).

@boegel
Copy link
Member

boegel commented Aug 6, 2024

Test report by @boegel
SUCCESS
Build succeeded for 3 out of 3 (3 easyconfigs in total)
node3153.skitty.os - Linux RHEL 8.8, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/boegel/8861d2a0eff3fbbdd2d4f9e9bd069027 for a full test report.

@boegel
Copy link
Member

boegel commented Aug 6, 2024

The top-level easyblock is copied over to the reprod directory and then the extensions are installed. The extensions also copy their easyblocks to the reprod directory, but they are using the same easyblock. This is fine when the permissions on the original file allow it to be overwritten, but when you start from a read-only installation of EasyBuild you end up with a permissions issue. If we switch to Bundle as the top-level easyblock we sidestep the problem.

The comment in easybuilders/easybuild-framework#4602 (comment) still stands, the framework could be a little more graceful here and recognise that the files have the same content (so the copy could be skipped).

OK makes sense now, thanks for clarifying!

@boegel
Copy link
Member

boegel commented Aug 6, 2024

Going in, thanks @ocaisa!

@boegel boegel merged commit dff5e3b into easybuilders:develop Aug 6, 2024
9 checks passed
@ocaisa ocaisa deleted the fix_bioperl branch August 6, 2024 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants