Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manta boost fix: add boost-cpp dependency at runtime. #53424

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

dslarm
Copy link
Contributor

@dslarm dslarm commented Jan 23, 2025

Describe your pull request here


Please read the guidelines for Bioconda recipes before opening a pull request (PR).

General instructions

  • If this PR adds or updates a recipe, use "Add" or "Update" appropriately as the first word in its title.
  • New recipes not directly relevant to the biological sciences need to be submitted to the conda-forge channel instead of Bioconda.
  • PRs require reviews prior to being merged. Once your PR is passing tests and ready to be merged, please issue the @BiocondaBot please add label command.
  • Please post questions on Gitter or ping @bioconda/core in a comment.

Instructions for avoiding API, ABI, and CLI breakage issues

Conda is able to record and lock (a.k.a. pin) dependency versions used at build time of other recipes.
This way, one can avoid that expectations of a downstream recipe with regards to API, ABI, or CLI are violated by later changes in the recipe.
If not already present in the meta.yaml, make sure to specify run_exports (see here for the rationale and comprehensive explanation).
Add a run_exports section like this:

build:
  run_exports:
    - ...

with ... being one of:

Case run_exports statement
semantic versioning {{ pin_subpackage("myrecipe", max_pin="x") }}
semantic versioning (0.x.x) {{ pin_subpackage("myrecipe", max_pin="x.x") }}
known breakage in minor versions {{ pin_subpackage("myrecipe", max_pin="x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
known breakage in patch versions {{ pin_subpackage("myrecipe", max_pin="x.x.x") }} (in such a case, please add a note that shortly mentions your evidence for that)
calendar versioning {{ pin_subpackage("myrecipe", max_pin=None) }}

while replacing "myrecipe" with either name if a name|lower variable is defined in your recipe or with the lowercase name of the package in quotes.

Bot commands for PR management

Please use the following BiocondaBot commands:

Everyone has access to the following BiocondaBot commands, which can be given in a comment:

@BiocondaBot please update Merge the master branch into a PR.
@BiocondaBot please add label Add the please review & merge label.
@BiocondaBot please fetch artifacts Post links to CI-built packages/containers.
You can use this to test packages locally.

Note that the @BiocondaBot please merge command is now depreciated. Please just squash and merge instead.

Also, the bot watches for comments from non-members that include @bioconda/<team> and will automatically re-post them to notify the addressed <team>.

@dslarm
Copy link
Contributor Author

dslarm commented Jan 23, 2025

@BiocondaBot please add label

@BiocondaBot BiocondaBot added the please review & merge set to ask for merge label Jan 23, 2025
@martin-g
Copy link
Contributor

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
linux-64 manta-1.6.0-py27h9948957_5.tar.bz2 linux-64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
linux-aarch64 manta-1.6.0-py27h163da20_5.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/5e61d620-77f2-48f5-804b-b0e0397a26b5/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
manta 1.6.0--py27h9948957_5 GitHub Actions
showImages are in the linux-64 zip file above.gzip -dc images/manta---1.6.0--py27h9948957_5.tar.gz | docker load

@martin-g
Copy link
Contributor

mgrigorov in 🌐 euler-arm-22 in /tmp/manta took 32s 
❯ tar xvf manta-1.6.0-py27h163da20_5.tar.bz2 
info/run_exports.json
info/test/run_test.sh
info/hash_input.json
info/index.json
info/recipe/build.sh
info/recipe/boost-shlibs.patch
info/recipe/meta.yaml.template
info/files
info/has_prefix
info/recipe/meta.yaml
info/recipe/conda_build_config.yaml
info/about.json
info/paths.json
info/git
share/demo/manta/data/G15512.HCC1954.1.COST16011_region.bam.bai
share/demo/manta/data/HCC1954.NORMAL.30x.compare.COST16011_region.bam.bai
share/demo/manta/expectedResults/somaticSV.vcf.gz
lib/python/configBuildTimeInfo.py
libexec/vcfCmdlineSwapper.py
libexec/sortBam.py
libexec/mergeBam.py
libexec/cat.py
libexec/mergeChromDepth.py
libexec/extractSmallIndelCandidates.py
libexec/sortEdgeLogs.py
libexec/updateSampleFTFilter.py
lib/python/estimateHardware.py
bin/runMantaWorkflowDemo.py
libexec/denovo_scoring.py
lib/python/pyflow/pyflowConfig.py
lib/python/sharedWorkflow.py
bin/configManta.py
lib/python/checkChromSet.py
libexec/ploidyFilter.py
libexec/sortVcf.py
lib/python/configureOptions.py
lib/python/makeRunScript.py
lib/python/configureUtil.py
lib/python/mantaOptions.py
libexec/convertInversion.py
lib/python/pyflow/pyflowTaskWrapper.py
lib/python/workflowUtil.py
lib/python/mantaWorkflow.py
lib/python/pyflow/pyflow.py
share/demo/manta/data/Homo_sapiens_assembly19.COST16011_region.fa
share/COPYRIGHT.txt
share/LICENSE.txt
bin/configManta.py.ini
share/demo/manta/data/Homo_sapiens_assembly19.COST16011_region.fa.fai
share/demo/manta/data/HCC1954.NORMAL.30x.compare.COST16011_region.bam
share/demo/manta/data/G15512.HCC1954.1.COST16011_region.bam
lib/python/configBuildTimeInfo.pyc
libexec/sortBam.pyc
libexec/vcfCmdlineSwapper.pyc
libexec/mergeBam.pyc
libexec/cat.pyc
libexec/mergeChromDepth.pyc
libexec/sortEdgeLogs.pyc
libexec/extractSmallIndelCandidates.pyc
libexec/denovo_scoring.pyc
libexec/updateSampleFTFilter.pyc
lib/python/estimateHardware.pyc
lib/python/pyflow/pyflowConfig.pyc
libexec/ploidyFilter.pyc
lib/python/sharedWorkflow.pyc
libexec/convertInversion.pyc
lib/python/checkChromSet.pyc
libexec/sortVcf.pyc
lib/python/mantaOptions.pyc
lib/python/configureOptions.pyc
lib/python/makeRunScript.pyc
lib/python/pyflow/pyflowTaskWrapper.pyc
lib/python/configureUtil.pyc
lib/python/workflowUtil.pyc
lib/python/mantaWorkflow.pyc
lib/python/pyflow/pyflow.pyc
share/demo/manta/README.md
share/CHANGELOG.md
libexec/bgzip
libexec/htsfile
libexec/tabix
libexec/samtools
libexec/SummarizeAlignmentStats
libexec/MergeAlignmentStats
libexec/TestAssembler
libexec/CheckSVLoci
libexec/SummarizeSVLoci
libexec/GetAlignmentStats
libexec/MergeSVLoci
libexec/GetChromDepth
libexec/DumpSVLoci
libexec/EstimateSVLoci
libexec/GenerateSVCandidates

mgrigorov in 🌐 euler-arm-22 in /tmp/manta took 6s 
❯ file bin/*
bin/configManta.py:          Python script, ASCII text executable
bin/configManta.py.ini:      ASCII text
bin/runMantaWorkflowDemo.py: Python script, ASCII text executable

mgrigorov in 🌐 euler-arm-22 in /tmp/manta 
❯ file libexec/*
libexec/bgzip:                           ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, not stripped
libexec/cat.py:                          Python script, ASCII text executable
libexec/cat.pyc:                         python 2.7 byte-compiled
libexec/CheckSVLoci:                     ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/convertInversion.py:             Python script, ASCII text executable
libexec/convertInversion.pyc:            python 2.7 byte-compiled
libexec/denovo_scoring.py:               Python script, ASCII text executable
libexec/denovo_scoring.pyc:              python 2.7 byte-compiled
libexec/DumpSVLoci:                      ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/EstimateSVLoci:                  ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/extractSmallIndelCandidates.py:  Python script, ASCII text executable
libexec/extractSmallIndelCandidates.pyc: python 2.7 byte-compiled
libexec/GenerateSVCandidates:            ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/GetAlignmentStats:               ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/GetChromDepth:                   ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/htsfile:                         ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, not stripped
libexec/MergeAlignmentStats:             ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/mergeBam.py:                     Python script, ASCII text executable
libexec/mergeBam.pyc:                    python 2.7 byte-compiled
libexec/mergeChromDepth.py:              Python script, ASCII text executable
libexec/mergeChromDepth.pyc:             python 2.7 byte-compiled
libexec/MergeSVLoci:                     ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/ploidyFilter.py:                 Python script, ASCII text executable
libexec/ploidyFilter.pyc:                python 2.7 byte-compiled
libexec/samtools:                        ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, not stripped
libexec/sortBam.py:                      Python script, ASCII text executable
libexec/sortBam.pyc:                     python 2.7 byte-compiled
libexec/sortEdgeLogs.py:                 Python script, ASCII text executable
libexec/sortEdgeLogs.pyc:                python 2.7 byte-compiled
libexec/sortVcf.py:                      Python script, ASCII text executable
libexec/sortVcf.pyc:                     python 2.7 byte-compiled
libexec/SummarizeAlignmentStats:         ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/SummarizeSVLoci:                 ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/tabix:                           ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, not stripped
libexec/TestAssembler:                   ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped
libexec/updateSampleFTFilter.py:         Python script, ASCII text executable
libexec/updateSampleFTFilter.pyc:        python 2.7 byte-compiled
libexec/vcfCmdlineSwapper.py:            Python script, ASCII text executable
libexec/vcfCmdlineSwapper.pyc:           python 2.7 byte-compiled

mgrigorov in 🌐 euler-arm-22 in /tmp/manta 
❯ ./bin/configManta.py
/usr/bin/env: ‘python2’: No such file or directory

mgrigorov in 🌐 euler-arm-22 in /tmp/manta 
❯ pixi init
✔ Created /tmp/manta/pixi.toml

mgrigorov in 🌐 euler-arm-22 in /tmp/manta 
❯ pixi add "python<3"
✔ Added python<3

mgrigorov in 🌐 euler-arm-22 in /tmp/manta took 15s 
❯ pixi shell

mgrigorov in 🌐 euler-arm-22 in /tmp/manta via 🅒 manta 
❯ ./bin/configManta.py                                                                                                                                                                                       (manta) 
Usage: configManta.py [options]

Version: 1.6.0

This script configures the Manta SV analysis pipeline.
You must specify a BAM or CRAM file for at least one sample.

Configuration will produce a workflow run script which
can execute the workflow on a single node or through
sge and resume any interrupted execution.

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --config=FILE         provide a configuration file to override defaults in
                        global config file (/tmp/manta/bin/configManta.py.ini)
  --allHelp             show all extended/hidden options

  Workflow options:
    --bam=FILE, --normalBam=FILE
                        Normal sample BAM or CRAM file. May be specified more
                        than once, multiple inputs will be treated as each BAM
                        file representing a different sample. [optional] (no
                        default)
    --tumorBam=FILE, --tumourBam=FILE
                        Tumor sample BAM or CRAM file. Only up to one tumor
                        bam file accepted. [optional] (no default)
    --exome             Set options for WES input: turn off depth filters
    --rna               Set options for RNA-Seq input. Must specify exactly
                        one bam input file
    --unstrandedRNA     Set if RNA-Seq input is unstranded: Allows splice-
                        junctions on either strand
    --referenceFasta=FILE
                        samtools-indexed reference fasta file [required]
    --runDir=DIR        Name of directory to be created where all workflow
                        scripts and output will be written. Each analysis
                        requires a separate directory. (default:
                        MantaWorkflow)
    --callRegions=FILE  Optionally provide a bgzip-compressed/tabix-indexed
                        BED file containing the set of regions to call. No VCF
                        output will be provided outside of these regions. The
                        full genome will still be used to estimate statistics
                        from the input (such as expected fragment size
                        distribution). Only one BED file may be specified.
                        (default: call the entire genome)

  Extended options (hidden):

mgrigorov in 🌐 euler-arm-22 in /tmp/manta via 🅒 manta 
❯ ./libexec/bgzip --help                                                                                                                                                                                     (manta) 

Version: 1.9
Usage:   bgzip [OPTIONS] [FILE] ...
Options:
   -b, --offset INT           decompress at virtual file pointer (0-based uncompressed offset)
   -c, --stdout               write on standard output, keep original files unchanged
   -d, --decompress           decompress
   -f, --force                overwrite files without asking
   -h, --help                 give this help
   -i, --index                compress and create BGZF index
   -I, --index-name FILE      name of BGZF index file [file.gz.gzi]
   -l, --compress-level INT   Compression level to use when compressing; 0 to 9, or -1 for default [-1]
   -r, --reindex              (re)index compressed file
   -g, --rebgzip              use an index file to bgzip a file
   -s, --size INT             decompress INT bytes (uncompressed size)
   -@, --threads INT          number of compression threads to use [1]
   -t, --test                 test integrity of compressed file

mgrigorov in 🌐 euler-arm-22 in /tmp/manta via 🅒 manta 
❯ ./libexec/GetChromDepth --help                                                                                                                                                                             (manta) 

GetChromDepth: get chromosome depth information from alignment files

version: 1.6.0
compiler: g++-13.3.0
build-time: 2025-01-23T09:24:12.564260Z

usage: GetChromDepth [options] [ > output ]

options:

configuration:
  --align-file arg       alignment file in BAM or CRAM format
  --chrom arg            chromosome name. May be supplied more than once. At 
                         least one entry required.
  --output-file arg      write stats to filename (default: stdout)
  --ref arg              fasta reference sequence (required)

help:
  -h [ --help ]          print this message



mgrigorov in 🌐 euler-arm-22 in /tmp/manta via 🅒 manta 
❯ cat pixi.toml                                                                                                                                                                                              (manta) 
[project]
authors = ["Martin Grigorov <[email protected]>"]
channels = ["conda-forge"]
description = "Add a short description here"
name = "manta"
platforms = ["linux-aarch64"]
version = "0.1.0"

[tasks]

[dependencies]
python = "<3"

It works for me even without installing boost-cpp ...

@dslarm
Copy link
Contributor Author

dslarm commented Jan 23, 2025

The current version didn't work for me w RHEL 9 - aarch64 or x86 - so I've added some post-install check to meta.yaml that should catch this.

@dslarm
Copy link
Contributor Author

dslarm commented Jan 23, 2025

@martin-g - can you review again and merge if all clear?

@martin-g
Copy link
Contributor

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
linux-64 manta-1.6.0-py27h9948957_5.tar.bz2 linux-64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
linux-aarch64 manta-1.6.0-py27h163da20_5.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/5f51f6db-ad60-4562-ba94-b01622800c67/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
manta 1.6.0--py27h9948957_5 GitHub Actions
showImages are in the linux-64 zip file above.gzip -dc images/manta---1.6.0--py27h9948957_5.tar.gz | docker load

@dslarm
Copy link
Contributor Author

dslarm commented Jan 24, 2025

Had to update the test: it would've passed even if having a missing library as that also prints out the executable name.
Reason for this not being a 'just run it' like the command it is the exit code for CheckSVLoci -h is 2 not 0 which I assume will be a fail.

@dslarm
Copy link
Contributor Author

dslarm commented Jan 24, 2025

09:59:34 BIOCONDA INFO (OUT)   File "/opt/conda/lib/python3.10/site-packages/conda_build/create_test.py", line 86, in create_shell_files
09:59:34 BIOCONDA INFO (OUT)     f.write(cmd)
09:59:34 BIOCONDA INFO (OUT) TypeError: write() argument must be str, not dict

It seems that a ':' in the string is sufficient to break bioconda's test mechanism.

@dslarm
Copy link
Contributor Author

dslarm commented Jan 24, 2025

@BiocondaBot please fetch artifacts

@BiocondaBot
Copy link
Collaborator

Package(s) built are ready for inspection:

Arch Package Zip File / Repodata CI Instructions
linux-64 manta-1.6.0-py27h9948957_5.tar.bz2 linux-64.zip GitHub Actions
showYou may also use conda to install after downloading and extracting the zip file. conda install -c ./packages <package name>
linux-aarch64 manta-1.6.0-py27h163da20_5.tar.bz2 repodata.json CircleCI
showYou may also use conda to install:conda install -c https://output.circle-artifacts.com/output/job/ca812115-87c2-4e98-9eb3-5549a8a77224/artifacts/0/tmp/artifacts/packages <package name>

Docker image(s) built:

Package Tag CI Install with docker
manta 1.6.0--py27h9948957_5 GitHub Actions
showImages are in the linux-64 zip file above.gzip -dc images/manta---1.6.0--py27h9948957_5.tar.gz | docker load

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
please review & merge set to ask for merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants