Skip to content

Commit

Permalink
adding MoP2
Browse files Browse the repository at this point in the history
lucacozzuto committed Sep 9, 2024
1 parent 2377840 commit 525b98f
Showing 168 changed files with 7,747 additions and 2,449 deletions.
40 changes: 33 additions & 7 deletions INSTALL.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,34 @@
wget https://mirror.oxfordnanoportal.com/software/analysis/ont-guppy_3.4.5_linux64.tar.gz
tar -zvxf ont-guppy_3.4.5_linux64.tar.gz
mkdir NanoPreprocess/bin/ont-guppy_3.4.5_linux64
mv ont-guppy NanoPreprocess/bin/ont-guppy_3.4.5_linux64
cd NanoPreprocess/bin
ln -s ont-guppy_3.4.5_linux64/ont-guppy/bin/guppy_* .
#!/bin/bash
#
# This script will download and install gouppy
# params:
# 1 guppy version

if [ x"$1" == x ]; then
GUPPY_VER='3.4.5'
else
GUPPY_VER=$1
fi


wget https://cdn.oxfordnanoportal.com/software/analysis/ont-guppy_${GUPPY_VER}_linux64.tar.gz
if [ $? -eq 0 ]; then
echo "INSTALLING GUPPY VERSION ${GUPPY_VER}"
else
echo "GUPPY VERSION ${GUPPY_VER} is not found"
exit
fi

tar -zvxf ont-guppy_${GUPPY_VER}_linux64.tar.gz
mv ont-guppy mop_preprocess/bin/
cd mop_preprocess/bin
ln -sf ont-guppy/bin/guppy_* .
ln -sf ont-guppy/lib/* .
cd ../../
rm -fr ont-guppy_3.4.5_linux64.tar.gz
if [ ! -e "mop_preprocess/bin/ont-guppy/lib/libz.so" ] ; then
unlink mop_preprocess/bin/ont-guppy/lib/libz.so
cd mop_preprocess/bin/ont-guppy/lib/
ln -sf libz.so.1 libz.so
cd ../../../../
fi
rm ont-guppy_${GUPPY_VER}_linux64.tar.gz
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2019 Biocore@CRG
Copyright (c) 2021 Biocore@CRG

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
63 changes: 36 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,58 @@
---
layout: page
title: Home
navigation: 1
---

# MoP2- DSL2 version of Master of Pores
[![Docker Build Status](https://img.shields.io/docker/automated/biocorecrg/nanopore.svg)](https://cloud.docker.com/u/biocorecrg/repository/docker/biocorecrg/nanopore/builds)
[![mop2-CI](https://github.com/biocorecrg/MoP2/actions/workflows/build.yml/badge.svg)](https://github.com/biocorecrg/MoP2/actions/workflows/build.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3518291.svg)](https://doi.org/10.5281/zenodo.3518291)
[![Nextflow version](https://img.shields.io/badge/Nextflow-21.04.1-brightgreen)](https://www.nextflow.io/)
[![Nextflow DSL2](https://img.shields.io/badge/Nextflow-DSL2-brightgreen)](https://www.nextflow.io/)
[![Singularity version](https://img.shields.io/badge/Singularity-v3.2.1-green.svg)](https://www.sylabs.io/)
[![Build Status](https://travis-ci.com/biocorecrg/master_of_pores.svg?branch=v1.1)](https://travis-ci.com/biocorecrg/master_of_pores)
[![Docker version](https://img.shields.io/badge/Docker-v19.03-blue)](https://www.docker.com/)
<img align="right" href="https://biocore.crg.eu/" src="https://raw.githubusercontent.com/CRG-CNAG/BioCoreMiscOpen/master/logo/biocore-logo_small.png" />
[![Docker version](https://img.shields.io/badge/Docker-v20.10.8-blue)](https://www.docker.com/)

<br/>

# ![Nanopore analysis pipeline](https://raw.githubusercontent.com/biocorecrg/master_of_pores/master/docs/logo_master.jpg)
![MOP2](https://github.com/biocorecrg/MoP2/blob/main/img/master_red.jpg?raw=true)

# Nanopore analysis pipeline
Nextflow pipeline for analysis of Nanopore data from direct RNA sequencing. This is a joint project between [CRG bioinformatics core](https://biocore.crg.eu/) and [Epitranscriptomics and RNA Dynamics research group](https://www.crg.eu/en/programmes-groups/novoa-lab).
The current version is the 1.1. The original 1.0 published in the paper is [here](https://github.com/biocorecrg/master_of_pores/releases/tag/v1.0)

-------------------------------------
## New DSL2 version of Master Of Pores is out!!
We translated the pipelines using DSL2 and added a new pipeline!! A lot of improvements and bug fixes!
[https://github.com/biocorecrg/MoP2](https://github.com/biocorecrg/MOP2)
Inspired by Metallica's [Master Of Puppets](https://www.youtube.com/watch?v=S7blkui3nQc)

---------------------------------
## Install
Please install nextflow and singularity or docker before.

Then download the repo:

## Documentation
Please read the documentation here: https://biocorecrg.github.io/master_of_pores/
```
git clone --depth 1 --recurse-submodules git@github.com:biocorecrg/MOP2.git
```

or
```
git clone --depth 1 --recurse-submodules https://github.com/biocorecrg/MOP2.git
```

You can use INSTALL.sh to download the version 3.4.5 of guppy or you can replace it with the version you prefer. Please consider that the support of VBZ compression of fast5 started with version 3.4.X.

```
cd MOP2; sh INSTALL.sh 3.4.5
```

## Testing
You can replace ```-with-singularity``` with ```-with-docker``` if you want to use the docker engine.

```
cd mop_preprocess
nextflow run mop_preprocess.nf -with-singularity -bg -profile local > log
```

## Reference
If you use this tool, please cite our paper:
If you use this tool, please cite our papers:

["Nanopore Direct RNA Sequencing Data Processing and Analysis Using MasterOfPores"
Cozzuto L, Delgado-Tejedor A, Hermoso Pulido T, Novoa EM, Ponomarenko J. *N. Methods Mol Biol. 2023*;2624:185-205. doi: 10.1007/978-1-0716-2962-8_13.](https://link.springer.com/protocol/10.1007/978-1-0716-2962-8_13)

["MasterOfPores: A Workflow for the Analysis of Oxford Nanopore Direct RNA Sequencing Datasets"
Luca Cozzuto, Huanle Liu, Leszek P. Pryszcz, Toni Hermoso Pulido, Anna Delgado-Tejedor, Julia Ponomarenko, Eva Maria Novoa.
*Front. Genet., 17 March 2020.* https://doi.org/10.3389/fgene.2020.00211](https://www.frontiersin.org/articles/10.3389/fgene.2020.00211/full)


## Contact

Please open an [issue](https://github.com/biocorecrg/master_of_pores/issues) if you encounter any issues/troubles.
However, please go over the previous issues (including closed issues) before opening a new issue, as your same exact question might have been already answered previously. Thank you!

## Documentation
The documentation is available at [https://biocorecrg.github.io/MOP2/docs/](https://biocorecrg.github.io/MOP2/docs/about.html)
6 changes: 6 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Missing

- [ ] A good documentation written as readthedocs
- [ ] Replacing Tombo with a faster option
- [ ] Improving the speed of the latest stage of mop_mod
- [ ] Add steps for assembly / quantification
Binary file removed anno/curlcake_constructs.fasta.gz
Binary file not shown.
Binary file added anno/yeast_rRNA_ref.fa.gz
Binary file not shown.
52 changes: 52 additions & 0 deletions conf/awsbatch.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
workDir = 's3://mop2-bucket-1/work'
aws.region = 'eu-central-1'
aws.batch.cliPath = '/home/ec2-user/miniconda/bin/aws'

process {
executor = 'awsbatch'
queue = 'mop'
cpus = 1
memory='2G'
cache='lenient'

container = 'biocorecrg/mopprepr:0.7'
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null}

withLabel: big_cpus {
cpus = 2
memory = '2G'
}

withLabel: big_cpus_ignore {
errorStrategy = 'ignore'
cpus = 2
memory = '2G'
}

withLabel: big_mem_cpus {
time = '6h'
cpus = 3
memory = '2G'
}

withLabel: big_time_cpus {
time = '24h'
cpus = 3
memory = '2G'
}

withLabel: demulti_gpus {
accelerator = 1
queue = 'mop-gpu'
memory = '2G'
containerOptions = { workflow.containerEngine == "singularity" ? '--nv':
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } }
withLabel: basecall_gpus {
accelerator = 1
queue = 'mop-gpu'
memory = '2G'
clusterOptions = '-l gpu=1'
containerOptions = { workflow.containerEngine == "singularity" ? '--nv':
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) }
}
}
47 changes: 47 additions & 0 deletions conf/awsbatch.config2
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
workDir = 's3://mop2-bucket-1/work'
aws.region = 'eu-central-1'
aws.batch.cliPath = '/home/ec2-user/miniconda/bin/aws'

process {
executor = 'awsbatch'
queue = 'mop'
cpus = 1
memory='2G'
cache='lenient'

container = 'biocorecrg/mopprepr:0.7'
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null}

withLabel: big_cpus {
cpus = 2
memory = '2G'
}

withLabel: big_cpus_ignore {
errorStrategy = 'ignore'
cpus = 2
memory = '2G'
}

withLabel: big_mem_cpus {
time = '6h'
cpus = 2
memory = '2G'
}

withLabel: demulti_gpus {
accelerator = 1
queue = 'mop-gpu'
memory = '2G'
// clusterOptions = {" -l gpu=1 -l virtual_free=${task.memory.toString().replaceAll(/[\sB]/,'')}"}
containerOptions = { workflow.containerEngine == "singularity" ? '--nv':
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) } }
withLabel: basecall_gpus {
accelerator = 1
queue = 'mop-gpu'
memory = '2G'
clusterOptions = '-l gpu=1'
containerOptions = { workflow.containerEngine == "singularity" ? '--nv':
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) }
}
}
12 changes: 12 additions & 0 deletions conf/ci.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
process {
executor = 'local'
cpus = 2
memory = '1.5GB'
cache='lenient'
container = 'biocorecrg/mopprepr:0.7'
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null}
withLabel: big_cpus_ignore {
errorStrategy = 'ignore'

}
}
17 changes: 17 additions & 0 deletions conf/local.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
process {
executor = 'local'
cpus = 1
memory = '1.5GB'
cache='lenient'
container = 'biocorecrg/mopprepr:0.7'
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null}
withLabel: big_cpus_ignore {
errorStrategy = 'ignore'

}
withLabel: basecall_gpus {
maxForks = 1
containerOptions = { workflow.containerEngine == "singularity" ? '--nv':
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) }
}
}
17 changes: 17 additions & 0 deletions conf/m1_apple.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
process {
executor = 'local'
cpus = 1
memory = '1.5GB'
cache='lenient'
container = 'biocorecrg/mopprepr:0.7'
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --platform linux/amd64': null}
withLabel: big_cpus_ignore {
errorStrategy = 'ignore'

}
withLabel: basecall_gpus {
maxForks = 1
containerOptions = { workflow.containerEngine == "singularity" ? '--nv':
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) }
}
}
70 changes: 70 additions & 0 deletions conf/sge.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
process {
executor = 'sge'
penv = 'smp'
queue = 'long-sl7,biocore-el7,short-sl7'
cpus = 1
memory='12G'
clusterOptions = { "-l h_vmem=${task.memory.toString().replaceAll(/[\sB]/,'')}" }
cache='lenient'
container = 'biocorecrg/mopprepr:0.7'
containerOptions = { workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g)': null}
withLabel: big_mem_time {
memory = '60G'
time = '24h'
}
withLabel: big_time_cpus {
cpus = 8
memory = '60G'
time = '24h'
clusterOptions = { "-l h_vmem=${task.memory.toString().replaceAll(/[\sB]/,'')}" }

}
withLabel: big_cpus {
cpus = 8
memory = '8G'
}
withLabel: big_mem {
cpus = 1
memory = '80G'
}
withLabel: big_cpus_retry {
errorStrategy = 'retry'
cpus = 8
memory = {8.GB * task.attempt}
time = {6.h * task.attempt}
maxRetries = 3

}
withLabel: big_cpus_ignore {
errorStrategy = 'ignore'
cpus = 8
memory = '8G'
}
withLabel: big_mem_cpus {
cpus = 8
memory = '60G'
}

withLabel: big_mem_cpus_ret {
cpus = 8
memory = '60G'
time = {6.h * task.attempt}
}

withLabel: demulti_gpus {
queue = 'gpu'
memory = '20G'
clusterOptions = {" -l gpu=1 -l virtual_free=${task.memory.toString().replaceAll(/[\sB]/,'')}"}
singularity.envWhitelist = "CUDA_VISIBLE_DEVICES,SGE_HGR_gpu"
containerOptions = { workflow.containerEngine == "singularity" ? '--nv':
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) }
}
withLabel: basecall_gpus {
queue = 'gpu'
memory = '8G'
clusterOptions = '-l gpu=1'
singularity.envWhitelist = "CUDA_VISIBLE_DEVICES,SGE_HGR_gpu"
containerOptions = { workflow.containerEngine == "singularity" ? '--nv':
( workflow.containerEngine == "docker" ? '-u $(id -u):$(id -g) --gpus all': null ) }
}
}
Loading

0 comments on commit 525b98f

Please sign in to comment.