-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
1 parent
2377840
commit 525b98f
Showing
168 changed files
with
7,747 additions
and
2,449 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,8 +1,34 @@ | ||
wget https://mirror.oxfordnanoportal.com/software/analysis/ont-guppy_3.4.5_linux64.tar.gz | ||
tar -zvxf ont-guppy_3.4.5_linux64.tar.gz | ||
mkdir NanoPreprocess/bin/ont-guppy_3.4.5_linux64 | ||
mv ont-guppy NanoPreprocess/bin/ont-guppy_3.4.5_linux64 | ||
cd NanoPreprocess/bin | ||
ln -s ont-guppy_3.4.5_linux64/ont-guppy/bin/guppy_* . | ||
#!/bin/bash | ||
# | ||
# This script will download and install gouppy | ||
# params: | ||
# 1 guppy version | ||
|
||
if [ x"$1" == x ]; then | ||
GUPPY_VER='3.4.5' | ||
else | ||
GUPPY_VER=$1 | ||
fi | ||
|
||
|
||
wget https://cdn.oxfordnanoportal.com/software/analysis/ont-guppy_${GUPPY_VER}_linux64.tar.gz | ||
if [ $? -eq 0 ]; then | ||
echo "INSTALLING GUPPY VERSION ${GUPPY_VER}" | ||
else | ||
echo "GUPPY VERSION ${GUPPY_VER} is not found" | ||
exit | ||
fi | ||
|
||
tar -zvxf ont-guppy_${GUPPY_VER}_linux64.tar.gz | ||
mv ont-guppy mop_preprocess/bin/ | ||
cd mop_preprocess/bin | ||
ln -sf ont-guppy/bin/guppy_* . | ||
ln -sf ont-guppy/lib/* . | ||
cd ../../ | ||
rm -fr ont-guppy_3.4.5_linux64.tar.gz | ||
if [ ! -e "mop_preprocess/bin/ont-guppy/lib/libz.so" ] ; then | ||
unlink mop_preprocess/bin/ont-guppy/lib/libz.so | ||
cd mop_preprocess/bin/ont-guppy/lib/ | ||
ln -sf libz.so.1 libz.so | ||
cd ../../../../ | ||
fi | ||
rm ont-guppy_${GUPPY_VER}_linux64.tar.gz |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,49 +1,58 @@ | ||
--- | ||
layout: page | ||
title: Home | ||
navigation: 1 | ||
--- | ||
|
||
# MoP2- DSL2 version of Master of Pores | ||
[![Docker Build Status](https://img.shields.io/docker/automated/biocorecrg/nanopore.svg)](https://cloud.docker.com/u/biocorecrg/repository/docker/biocorecrg/nanopore/builds) | ||
[![mop2-CI](https://github.com/biocorecrg/MoP2/actions/workflows/build.yml/badge.svg)](https://github.com/biocorecrg/MoP2/actions/workflows/build.yml) | ||
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) | ||
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3518291.svg)](https://doi.org/10.5281/zenodo.3518291) | ||
[![Nextflow version](https://img.shields.io/badge/Nextflow-21.04.1-brightgreen)](https://www.nextflow.io/) | ||
[![Nextflow DSL2](https://img.shields.io/badge/Nextflow-DSL2-brightgreen)](https://www.nextflow.io/) | ||
[![Singularity version](https://img.shields.io/badge/Singularity-v3.2.1-green.svg)](https://www.sylabs.io/) | ||
[![Build Status](https://travis-ci.com/biocorecrg/master_of_pores.svg?branch=v1.1)](https://travis-ci.com/biocorecrg/master_of_pores) | ||
[![Docker version](https://img.shields.io/badge/Docker-v19.03-blue)](https://www.docker.com/) | ||
<img align="right" href="https://biocore.crg.eu/" src="https://raw.githubusercontent.com/CRG-CNAG/BioCoreMiscOpen/master/logo/biocore-logo_small.png" /> | ||
[![Docker version](https://img.shields.io/badge/Docker-v20.10.8-blue)](https://www.docker.com/) | ||
|
||
<br/> | ||
|
||
# ![Nanopore analysis pipeline](https://raw.githubusercontent.com/biocorecrg/master_of_pores/master/docs/logo_master.jpg) | ||
![MOP2](https://github.com/biocorecrg/MoP2/blob/main/img/master_red.jpg?raw=true) | ||
|
||
# Nanopore analysis pipeline | ||
Nextflow pipeline for analysis of Nanopore data from direct RNA sequencing. This is a joint project between [CRG bioinformatics core](https://biocore.crg.eu/) and [Epitranscriptomics and RNA Dynamics research group](https://www.crg.eu/en/programmes-groups/novoa-lab). | ||
The current version is the 1.1. The original 1.0 published in the paper is [here](https://github.com/biocorecrg/master_of_pores/releases/tag/v1.0) | ||
|
||
------------------------------------- | ||
## New DSL2 version of Master Of Pores is out!! | ||
We translated the pipelines using DSL2 and added a new pipeline!! A lot of improvements and bug fixes! | ||
[https://github.com/biocorecrg/MoP2](https://github.com/biocorecrg/MOP2) | ||
Inspired by Metallica's [Master Of Puppets](https://www.youtube.com/watch?v=S7blkui3nQc) | ||
|
||
--------------------------------- | ||
## Install | ||
Please install nextflow and singularity or docker before. | ||
|
||
Then download the repo: | ||
|
||
## Documentation | ||
Please read the documentation here: https://biocorecrg.github.io/master_of_pores/ | ||
``` | ||
git clone --depth 1 --recurse-submodules git@github.com:biocorecrg/MOP2.git | ||
``` | ||
|
||
or | ||
``` | ||
git clone --depth 1 --recurse-submodules https://github.com/biocorecrg/MOP2.git | ||
``` | ||
|
||
You can use INSTALL.sh to download the version 3.4.5 of guppy or you can replace it with the version you prefer. Please consider that the support of VBZ compression of fast5 started with version 3.4.X. | ||
|
||
``` | ||
cd MOP2; sh INSTALL.sh 3.4.5 | ||
``` | ||
|
||
## Testing | ||
You can replace ```-with-singularity``` with ```-with-docker``` if you want to use the docker engine. | ||
|
||
``` | ||
cd mop_preprocess | ||
nextflow run mop_preprocess.nf -with-singularity -bg -profile local > log | ||
``` | ||
|
||
## Reference | ||
If you use this tool, please cite our paper: | ||
If you use this tool, please cite our papers: | ||
|
||
["Nanopore Direct RNA Sequencing Data Processing and Analysis Using MasterOfPores" | ||
Cozzuto L, Delgado-Tejedor A, Hermoso Pulido T, Novoa EM, Ponomarenko J. *N. Methods Mol Biol. 2023*;2624:185-205. doi: 10.1007/978-1-0716-2962-8_13.](https://link.springer.com/protocol/10.1007/978-1-0716-2962-8_13) | ||
|
||
["MasterOfPores: A Workflow for the Analysis of Oxford Nanopore Direct RNA Sequencing Datasets" | ||
Luca Cozzuto, Huanle Liu, Leszek P. Pryszcz, Toni Hermoso Pulido, Anna Delgado-Tejedor, Julia Ponomarenko, Eva Maria Novoa. | ||
*Front. Genet., 17 March 2020.* https://doi.org/10.3389/fgene.2020.00211](https://www.frontiersin.org/articles/10.3389/fgene.2020.00211/full) | ||
|
||
|
||
## Contact | ||
|
||
Please open an [issue](https://github.com/biocorecrg/master_of_pores/issues) if you encounter any issues/troubles. | ||
However, please go over the previous issues (including closed issues) before opening a new issue, as your same exact question might have been already answered previously. Thank you! | ||
|
||
## Documentation | ||
The documentation is available at [https://biocorecrg.github.io/MOP2/docs/](https://biocorecrg.github.io/MOP2/docs/about.html) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
# Missing | ||
|
||
- [ ] A good documentation written as readthedocs | ||
- [ ] Replacing Tombo with a faster option | ||
- [ ] Improving the speed of the latest stage of mop_mod | ||
- [ ] Add steps for assembly / quantification |
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
workDir = 's3://mop2-bucket-1/work' | ||
aws.region = 'eu-central-1' | ||
aws.batch.cliPath = '/home/ec2-user/miniconda/bin/aws' | ||
|
||
process { | ||
executor = 'awsbatch' | ||
queue = 'mop' | ||
cpus = 1 | ||
memory='2G' | ||
cache='lenient' | ||
|
||
container = 'biocorecrg/mopprepr:0.7' | ||
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null} | ||
|
||
withLabel: big_cpus { | ||
cpus = 2 | ||
memory = '2G' | ||
} | ||
|
||
withLabel: big_cpus_ignore { | ||
errorStrategy = 'ignore' | ||
cpus = 2 | ||
memory = '2G' | ||
} | ||
|
||
withLabel: big_mem_cpus { | ||
time = '6h' | ||
cpus = 3 | ||
memory = '2G' | ||
} | ||
|
||
withLabel: big_time_cpus { | ||
time = '24h' | ||
cpus = 3 | ||
memory = '2G' | ||
} | ||
|
||
withLabel: demulti_gpus { | ||
accelerator = 1 | ||
queue = 'mop-gpu' | ||
memory = '2G' | ||
containerOptions = { workflow.containerEngine == "singularity" ? '--nv': | ||
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } } | ||
withLabel: basecall_gpus { | ||
accelerator = 1 | ||
queue = 'mop-gpu' | ||
memory = '2G' | ||
clusterOptions = '-l gpu=1' | ||
containerOptions = { workflow.containerEngine == "singularity" ? '--nv': | ||
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
workDir = 's3://mop2-bucket-1/work' | ||
aws.region = 'eu-central-1' | ||
aws.batch.cliPath = '/home/ec2-user/miniconda/bin/aws' | ||
|
||
process { | ||
executor = 'awsbatch' | ||
queue = 'mop' | ||
cpus = 1 | ||
memory='2G' | ||
cache='lenient' | ||
|
||
container = 'biocorecrg/mopprepr:0.7' | ||
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null} | ||
|
||
withLabel: big_cpus { | ||
cpus = 2 | ||
memory = '2G' | ||
} | ||
|
||
withLabel: big_cpus_ignore { | ||
errorStrategy = 'ignore' | ||
cpus = 2 | ||
memory = '2G' | ||
} | ||
|
||
withLabel: big_mem_cpus { | ||
time = '6h' | ||
cpus = 2 | ||
memory = '2G' | ||
} | ||
|
||
withLabel: demulti_gpus { | ||
accelerator = 1 | ||
queue = 'mop-gpu' | ||
memory = '2G' | ||
// clusterOptions = {" -l gpu=1 -l virtual_free=${task.memory.toString().replaceAll(/[\sB]/,'')}"} | ||
containerOptions = { workflow.containerEngine == "singularity" ? '--nv': | ||
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } } | ||
withLabel: basecall_gpus { | ||
accelerator = 1 | ||
queue = 'mop-gpu' | ||
memory = '2G' | ||
clusterOptions = '-l gpu=1' | ||
containerOptions = { workflow.containerEngine == "singularity" ? '--nv': | ||
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
process { | ||
executor = 'local' | ||
cpus = 2 | ||
memory = '1.5GB' | ||
cache='lenient' | ||
container = 'biocorecrg/mopprepr:0.7' | ||
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null} | ||
withLabel: big_cpus_ignore { | ||
errorStrategy = 'ignore' | ||
|
||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
process { | ||
executor = 'local' | ||
cpus = 1 | ||
memory = '1.5GB' | ||
cache='lenient' | ||
container = 'biocorecrg/mopprepr:0.7' | ||
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null} | ||
withLabel: big_cpus_ignore { | ||
errorStrategy = 'ignore' | ||
|
||
} | ||
withLabel: basecall_gpus { | ||
maxForks = 1 | ||
containerOptions = { workflow.containerEngine == "singularity" ? '--nv': | ||
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
process { | ||
executor = 'local' | ||
cpus = 1 | ||
memory = '1.5GB' | ||
cache='lenient' | ||
container = 'biocorecrg/mopprepr:0.7' | ||
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --platform linux/amd64': null} | ||
withLabel: big_cpus_ignore { | ||
errorStrategy = 'ignore' | ||
|
||
} | ||
withLabel: basecall_gpus { | ||
maxForks = 1 | ||
containerOptions = { workflow.containerEngine == "singularity" ? '--nv': | ||
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
process { | ||
executor = 'sge' | ||
penv = 'smp' | ||
queue = 'long-sl7,biocore-el7,short-sl7' | ||
cpus = 1 | ||
memory='12G' | ||
clusterOptions = { "-l h_vmem=${task.memory.toString().replaceAll(/[\sB]/,'')}" } | ||
cache='lenient' | ||
container = 'biocorecrg/mopprepr:0.7' | ||
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null} | ||
withLabel: big_mem_time { | ||
memory = '60G' | ||
time = '24h' | ||
} | ||
withLabel: big_time_cpus { | ||
cpus = 8 | ||
memory = '60G' | ||
time = '24h' | ||
clusterOptions = { "-l h_vmem=${task.memory.toString().replaceAll(/[\sB]/,'')}" } | ||
|
||
} | ||
withLabel: big_cpus { | ||
cpus = 8 | ||
memory = '8G' | ||
} | ||
withLabel: big_mem { | ||
cpus = 1 | ||
memory = '80G' | ||
} | ||
withLabel: big_cpus_retry { | ||
errorStrategy = 'retry' | ||
cpus = 8 | ||
memory = {8.GB * task.attempt} | ||
time = {6.h * task.attempt} | ||
maxRetries = 3 | ||
|
||
} | ||
withLabel: big_cpus_ignore { | ||
errorStrategy = 'ignore' | ||
cpus = 8 | ||
memory = '8G' | ||
} | ||
withLabel: big_mem_cpus { | ||
cpus = 8 | ||
memory = '60G' | ||
} | ||
|
||
withLabel: big_mem_cpus_ret { | ||
cpus = 8 | ||
memory = '60G' | ||
time = {6.h * task.attempt} | ||
} | ||
|
||
withLabel: demulti_gpus { | ||
queue = 'gpu' | ||
memory = '20G' | ||
clusterOptions = {" -l gpu=1 -l virtual_free=${task.memory.toString().replaceAll(/[\sB]/,'')}"} | ||
singularity.envWhitelist = "CUDA_VISIBLE_DEVICES,SGE_HGR_gpu" | ||
containerOptions = { workflow.containerEngine == "singularity" ? '--nv': | ||
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } | ||
} | ||
withLabel: basecall_gpus { | ||
queue = 'gpu' | ||
memory = '8G' | ||
clusterOptions = '-l gpu=1' | ||
singularity.envWhitelist = "CUDA_VISIBLE_DEVICES,SGE_HGR_gpu" | ||
containerOptions = { workflow.containerEngine == "singularity" ? '--nv': | ||
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } | ||
} | ||
} |
Oops, something went wrong.