Skip to content

Error while downloading docker image and tools - non-zero exit status 1 #189

New issue
New issue
Open
Open
Error while downloading docker image and tools - non-zero exit status 1#189
@sarthi2395

Description

@sarthi2395
sarthi2395
opened on May 27, 2020

I am using the installation instructions provided on the homepage to install bcbio-vm. I have completed the setup till adding the docker group. I am not able to install the docker image, the genome data and the tools. Looks like the download is fine, but something goes wrong during installation.

The log:

bcbio_vm.py --datadir=~/install/bcbio-vm/data install --data --tools --genomes GRCh37 --aligners bwa
Retrieving bcbio-nextgen docker image with code and tools
Using default tag: latest
latest: Pulling from bcbio/bcbio-vc
Digest: sha256:97bcadba81529438ecb3ffb703211f0b6a80d9e252512312d684d68bbda45b75
Status: Image is up to date for quay.io/bcbio/bcbio-vc:latest
quay.io/bcbio/bcbio-vc:latest
[2020-05-27T06:45Z] DEBUG: Running in docker container: 870fa26ee434a6522f800818                                                                                        ec33cf2a7804f6c9dc502fd719ce2c43c3221d46
[2020-05-27T06:45Z] DEBUG: docker attach --no-stdin 870fa26ee434a6522f800818ec33                                                                                        cf2a7804f6c9dc502fd719ce2c43c3221d46
[2020-05-27T06:45Z] DEBUG: Upgrading bcbio
[2020-05-27T06:45Z] DEBUG: Upgrade completed successfully.
[2020-05-27T06:45Z] DEBUG: Running in docker container: 12d33b4f2f096c192aad42a7                                                                                        73fdad0abe0b6d86e71ec75b057939918ec41691
[2020-05-27T06:45Z] DEBUG: docker attach --no-stdin 12d33b4f2f096c192aad42a773fd                                                                                        ad0abe0b6d86e71ec75b057939918ec41691
[2020-05-27T06:45Z] DEBUG: --2020-05-27 06:45:28--  https://github.com/chapmanb/                                                                                        cloudbiolinux/archive/master.tar.gz
[2020-05-27T06:45Z] DEBUG: Resolving github.com (github.com)... 13.234.210.38
[2020-05-27T06:45Z] DEBUG: Connecting to github.com (github.com)|13.234.210.38|:                                                                                        443... connected.
[2020-05-27T06:45Z] DEBUG: HTTP request sent, awaiting response... 302 Found
[2020-05-27T06:45Z] DEBUG: Location: https://codeload.github.com/chapmanb/cloudb                                                                                        iolinux/tar.gz/master [following]
[2020-05-27T06:45Z] DEBUG: --2020-05-27 06:45:28--  https://codeload.github.com/                                                                                        chapmanb/cloudbiolinux/tar.gz/master
[2020-05-27T06:45Z] DEBUG: Resolving codeload.github.com (codeload.github.com)..                                                                                        . 13.127.152.42
[2020-05-27T06:45Z] DEBUG: Connecting to codeload.github.com (codeload.github.co                                                                                        m)|13.127.152.42|:443... connected.
[2020-05-27T06:45Z] DEBUG: HTTP request sent, awaiting response... 200 OK
[2020-05-27T06:45Z] DEBUG: Length: unspecified [application/x-gzip]
[2020-05-27T06:45Z] DEBUG: Saving to: 'STDOUT'
[2020-05-27T06:45Z] DEBUG:      0K ........ ........ ........ ........ ........                                                                                         ........ 6.95M
[2020-05-27T06:45Z] DEBUG:   3072K ........ ........ ........ .......                                                                                                            7.06M=0.7s
[2020-05-27T06:45Z] DEBUG: 2020-05-27 06:45:30 (6.99 MB/s) - written to stdout [                                                                                        5189878]
[2020-05-27T06:45Z] DEBUG: --2020-05-27 06:45:30--  http://ftp.ncbi.nih.gov/snp/                                                                                        pre_build152/organisms/human_9606_b151_GRCh37p13/VCF/GATK/All_20180423.vcf.gz
[2020-05-27T06:45Z] DEBUG: Resolving ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)... 130.14.250.7, 2607:f220:41e:250::11
[2020-05-27T06:45Z] DEBUG: Connecting to ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|130.14.250.7|:80... connected.
[2020-05-27T06:45Z] DEBUG: HTTP request sent, awaiting response... 301 Moved Permanently
[2020-05-27T06:45Z] DEBUG: Location: https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b151_GRCh37p13/VCF/GATK/All_20180423.vcf.gz [following]
[2020-05-27T06:45Z] DEBUG: --2020-05-27 06:45:35--  https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b151_GRCh37p13/VCF/GATK/All_20180423.vcf.gz
[2020-05-27T06:45Z] DEBUG: Connecting to ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|130.14.250.7|:443... connected.
[2020-05-27T06:45Z] DEBUG: HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable
[2020-05-27T06:45Z] DEBUG:     The file is already fully retrieved; nothing to do.
[2020-05-27T07:00Z] DEBUG: [W::bgzf_read_block] EOF marker is absent. The input is probably truncated
[2020-05-27T07:00Z] DEBUG: tbx_index_build failed: variation/dbsnp-151.vcf.gz
[2020-05-27T07:00Z] DEBUG: Upgrading bcbio
[2020-05-27T07:00Z] DEBUG: Upgrading bcbio-nextgen data files
[2020-05-27T07:00Z] DEBUG: List of genomes to get (from the config file at '{'genomes': [{'dbkey': 'GRCh37', 'name': 'Human (GRCh37)', 'indexes': ['seq', 'twobit'], 'annotations': ['GA4GH_problem_regions', 'capture_regions', 'MIG', 'prioritize', 'dbsnp', 'hapmap', '1000g_omni_snps', 'ACMG56_genes', '1000g_snps', 'mills_indels', 'clinvar', 'cosmic', 'ancestral', 'qsignature', 'genesplicer', 'effects_transcripts', 'varpon', 'vcfanno', 'viral', 'transcripts', 'RADAR', 'fusion-blacklist', 'mirbase'], 'validation': ['giab-NA12878', 'giab-NA24385', 'giab-NA24631', 'dream-syn3', 'dream-syn4', 'giab-NA12878-NA24385-somatic', 'giab-NA24143', 'giab-NA24149', 'giab-NA24694', 'giab-NA24695']}], 'genome_indexes': ['bwa', 'rtg'], 'install_liftover': False, 'install_uniref': False}'): Human (GRCh37)
[2020-05-27T07:00Z] DEBUG: Running GGD recipe: GRCh37 dbsnp 151-20180423
[2020-05-27T07:00Z] DEBUG: Traceback (most recent call last):
[2020-05-27T07:00Z] DEBUG:   File "/usr/local/bin/bcbio_nextgen.py", line 228, in <module>
[2020-05-27T07:00Z] DEBUG:     install.upgrade_bcbio(kwargs["args"])
[2020-05-27T07:00Z] DEBUG:   File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 107, in upgrade_bcbio
[2020-05-27T07:00Z] DEBUG:     upgrade_bcbio_data(args, REMOTES)
[2020-05-27T07:00Z] DEBUG:   File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 377, in upgrade_bcbio_data
[2020-05-27T07:00Z] DEBUG:     args.cores, ["ggd", "s3", "raw"])
[2020-05-27T07:00Z] DEBUG:   File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 354, in install_data_local
[2020-05-27T07:00Z] DEBUG:     _prep_genomes(env, genomes, genome_indexes, ready_approaches, data_filedir)
[2020-05-27T07:00Z] DEBUG:   File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 480, in _prep_genomes
[2020-05-27T07:00Z] DEBUG:     retrieve_fn(env, manager, gid, idx)
[2020-05-27T07:00Z] DEBUG:   File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 875, in _install_with_ggd
[2020-05-27T07:00Z] DEBUG:     ggd.install_recipe(os.getcwd(), env.system_install, recipe_file, gid)
[2020-05-27T07:00Z] DEBUG:   File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 30, in install_recipe
[2020-05-27T07:00Z] DEBUG:     recipe["recipe"]["full"]["recipe_type"], system_install)
[2020-05-27T07:00Z] DEBUG:   File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 62, in _run_recipe
[2020-05-27T07:00Z] DEBUG:     subprocess.check_output(["bash", run_file])
[2020-05-27T07:00Z] DEBUG:   File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/subprocess.py", line 356, in check_output
[2020-05-27T07:00Z] DEBUG:     **kwargs).stdout
[2020-05-27T07:00Z] DEBUG:   File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/subprocess.py", line 438, in run
[2020-05-27T07:00Z] DEBUG:     output=stdout, stderr=stderr)
[2020-05-27T07:00Z] DEBUG: subprocess.CalledProcessError: Command '['bash', '/mnt/biodata/genomes/Hsapiens/GRCh37/txtmp/ggd-run.sh']' returned non-zero exit status 1.
[2020-05-27T07:00Z] ERROR: Uncaught exception occurred
Traceback (most recent call last):
  File "/home/zuser/install/bcbio-vm/anaconda/lib/python3.7/site-packages/bcbio/provenance/do.py", line 26, in run
    _do_run(cmd, checks, log_stdout, env=env)
  File "/home/zuser/install/bcbio-vm/anaconda/lib/python3.7/site-packages/bcbio/provenance/do.py", line 106, in _do_run
    raise subprocess.CalledProcessError(exitcode, error_msg)
subprocess.CalledProcessError: Command 'docker attach --no-stdin 12d33b4f2f096c192aad42a773fdad0abe0b6d86e71ec75b057939918ec41691
--2020-05-27 06:45:28--  https://github.com/chapmanb/cloudbiolinux/archive/master.tar.gz
Resolving github.com (github.com)... 13.234.210.38
Connecting to github.com (github.com)|13.234.210.38|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master [following]
--2020-05-27 06:45:28--  https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master
Resolving codeload.github.com (codeload.github.com)... 13.127.152.42
Connecting to codeload.github.com (codeload.github.com)|13.127.152.42|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: 'STDOUT'
     0K ........ ........ ........ ........ ........ ........ 6.95M
  3072K ........ ........ ........ .......                    7.06M=0.7s
2020-05-27 06:45:30 (6.99 MB/s) - written to stdout [5189878]
--2020-05-27 06:45:30--  http://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b151_GRCh37p13/VCF/GATK/All_20180423.vcf.gz
Resolving ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)... 130.14.250.7, 2607:f220:41e:250::11
Connecting to ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|130.14.250.7|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b151_GRCh37p13/VCF/GATK/All_20180423.vcf.gz [following]
--2020-05-27 06:45:35--  https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b151_GRCh37p13/VCF/GATK/All_20180423.vcf.gz
Connecting to ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|130.14.250.7|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable
    The file is already fully retrieved; nothing to do.
[W::bgzf_read_block] EOF marker is absent. The input is probably truncated
tbx_index_build failed: variation/dbsnp-151.vcf.gz
Upgrading bcbio
Upgrading bcbio-nextgen data files
List of genomes to get (from the config file at '{'genomes': [{'dbkey': 'GRCh37', 'name': 'Human (GRCh37)', 'indexes': ['seq', 'twobit'], 'annotations': ['GA4GH_problem_regions', 'capture_regions', 'MIG', 'prioritize', 'dbsnp', 'hapmap', '1000g_omni_snps', 'ACMG56_genes', '1000g_snps', 'mills_indels', 'clinvar', 'cosmic', 'ancestral', 'qsignature', 'genesplicer', 'effects_transcripts', 'varpon', 'vcfanno', 'viral', 'transcripts', 'RADAR', 'fusion-blacklist', 'mirbase'], 'validation': ['giab-NA12878', 'giab-NA24385', 'giab-NA24631', 'dream-syn3', 'dream-syn4', 'giab-NA12878-NA24385-somatic', 'giab-NA24143', 'giab-NA24149', 'giab-NA24694', 'giab-NA24695']}], 'genome_indexes': ['bwa', 'rtg'], 'install_liftover': False, 'install_uniref': False}'): Human (GRCh37)
Running GGD recipe: GRCh37 dbsnp 151-20180423
Traceback (most recent call last):
  File "/usr/local/bin/bcbio_nextgen.py", line 228, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 107, in upgrade_bcbio
    upgrade_bcbio_data(args, REMOTES)
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 377, in upgrade_bcbio_data
    args.cores, ["ggd", "s3", "raw"])
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 354, in install_data_local
    _prep_genomes(env, genomes, genome_indexes, ready_approaches, data_filedir)
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 480, in _prep_genomes
    retrieve_fn(env, manager, gid, idx)
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 875, in _install_with_ggd
    ggd.install_recipe(os.getcwd(), env.system_install, recipe_file, gid)
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 30, in install_recipe
    recipe["recipe"]["full"]["recipe_type"], system_install)
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 62, in _run_recipe
    subprocess.check_output(["bash", run_file])
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bash', '/mnt/biodata/genomes/Hsapiens/GRCh37/txtmp/ggd-run.sh']' returned non-zero exit status 1.
' returned non-zero exit status 1.
Stopping docker container
Traceback (most recent call last):
  File "/usr/local/bin/bcbio_vm.py", line 354, in <module>
    args.func(args)
  File "/usr/local/bin/bcbio_vm.py", line 40, in cmd_install
    install.full(args, devel.DOCKER)
  File "/home/zuser/install/bcbio-vm/anaconda/lib/python3.7/site-packages/bcbiovm/docker/install.py", line 40, in full
    manage.run_bcbio_cmd(args.image, dmounts, _get_cl(args))
  File "/home/zuser/install/bcbio-vm/anaconda/lib/python3.7/site-packages/bcbiovm/docker/manage.py", line 48, in run_bcbio_cmd
    raise e
  File "/home/zuser/install/bcbio-vm/anaconda/lib/python3.7/site-packages/bcbiovm/docker/manage.py", line 44, in run_bcbio_cmd
    log_stdout=True)
  File "/home/zuser/install/bcbio-vm/anaconda/lib/python3.7/site-packages/bcbio/provenance/do.py", line 26, in run
    _do_run(cmd, checks, log_stdout, env=env)
  File "/home/zuser/install/bcbio-vm/anaconda/lib/python3.7/site-packages/bcbio/provenance/do.py", line 106, in _do_run
    raise subprocess.CalledProcessError(exitcode, error_msg)
subprocess.CalledProcessError: Command 'docker attach --no-stdin 12d33b4f2f096c192aad42a773fdad0abe0b6d86e71ec75b057939918ec41691
--2020-05-27 06:45:28--  https://github.com/chapmanb/cloudbiolinux/archive/master.tar.gz
Resolving github.com (github.com)... 13.234.210.38
Connecting to github.com (github.com)|13.234.210.38|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master [following]
--2020-05-27 06:45:28--  https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master
Resolving codeload.github.com (codeload.github.com)... 13.127.152.42
Connecting to codeload.github.com (codeload.github.com)|13.127.152.42|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: 'STDOUT'
     0K ........ ........ ........ ........ ........ ........ 6.95M
  3072K ........ ........ ........ .......                    7.06M=0.7s
2020-05-27 06:45:30 (6.99 MB/s) - written to stdout [5189878]
--2020-05-27 06:45:30--  http://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b151_GRCh37p13/VCF/GATK/All_20180423.vcf.gz
Resolving ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)... 130.14.250.7, 2607:f220:41e:250::11
Connecting to ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|130.14.250.7|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b151_GRCh37p13/VCF/GATK/All_20180423.vcf.gz [following]
--2020-05-27 06:45:35--  https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/human_9606_b151_GRCh37p13/VCF/GATK/All_20180423.vcf.gz
Connecting to ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|130.14.250.7|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable
    The file is already fully retrieved; nothing to do.
[W::bgzf_read_block] EOF marker is absent. The input is probably truncated
tbx_index_build failed: variation/dbsnp-151.vcf.gz
Upgrading bcbio
Upgrading bcbio-nextgen data files
List of genomes to get (from the config file at '{'genomes': [{'dbkey': 'GRCh37', 'name': 'Human (GRCh37)', 'indexes': ['seq', 'twobit'], 'annotations': ['GA4GH_problem_regions', 'capture_regions', 'MIG', 'prioritize', 'dbsnp', 'hapmap', '1000g_omni_snps', 'ACMG56_genes', '1000g_snps', 'mills_indels', 'clinvar', 'cosmic', 'ancestral', 'qsignature', 'genesplicer', 'effects_transcripts', 'varpon', 'vcfanno', 'viral', 'transcripts', 'RADAR', 'fusion-blacklist', 'mirbase'], 'validation': ['giab-NA12878', 'giab-NA24385', 'giab-NA24631', 'dream-syn3', 'dream-syn4', 'giab-NA12878-NA24385-somatic', 'giab-NA24143', 'giab-NA24149', 'giab-NA24694', 'giab-NA24695']}], 'genome_indexes': ['bwa', 'rtg'], 'install_liftover': False, 'install_uniref': False}'): Human (GRCh37)
Running GGD recipe: GRCh37 dbsnp 151-20180423
Traceback (most recent call last):
  File "/usr/local/bin/bcbio_nextgen.py", line 228, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 107, in upgrade_bcbio
    upgrade_bcbio_data(args, REMOTES)
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 377, in upgrade_bcbio_data
    args.cores, ["ggd", "s3", "raw"])
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 354, in install_data_local
    _prep_genomes(env, genomes, genome_indexes, ready_approaches, data_filedir)
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 480, in _prep_genomes
    retrieve_fn(env, manager, gid, idx)
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 875, in _install_with_ggd
    ggd.install_recipe(os.getcwd(), env.system_install, recipe_file, gid)
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 30, in install_recipe
    recipe["recipe"]["full"]["recipe_type"], system_install)
  File "/home/zuser/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 62, in _run_recipe
    subprocess.check_output(["bash", run_file])
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/local/share/bcbio-nextgen/anaconda/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bash', '/mnt/biodata/genomes/Hsapiens/GRCh37/txtmp/ggd-run.sh']' returned non-zero exit status 1.
' returned non-zero exit status 1.

Could anyone help me out?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions