Simulation pipeline to generate next generation sequencing reads from realistic phylogenies. Can be used to test effects of model of evolution, rates of evolution, genomic distribution of mutations, and phylogenetic relatedness of samples and of reference genome on SNP calling and evolutionary inference.
TreeToReads (TTR) provides features not available in alternative simulation approaches:
- An observed genome is a tip on the simulated tree, allowing for direct comparison of read mapping between simulated and observed data.
- Variable sites are simulated on an input phylogeny, and next generation sequencing reads are simulated from these evolved genomes.
Together these make TTR a useful simulator for testing the effects of phylogenetic structure on read mapping and inference of variable sites.
Inputs are a phylogeny, a genome to be used as a tip in the tree, and a set of configuration parameters in a control file
Optional inputs can include a sequencing error model parameterized from empirical data, and a distribution for the distances separating pairs of mutations in the genome.
Outputs are mutated genomes representing all tips in the phylogeny, and simulated whole genome sequencing reads representing those genomes. These are are useful for testing and comparison of analysis pipelines. Mutations are currently only single nucleotide variants - no indels or rearrangements.
The code is still in development - but testing welcome, and will be supported via email ejmctavish, gmail.
- Seq-Gen
- Art
(can be run without Art if you want to generate mutated genomes, but not reads)
python packages
- Dendropy
git clone https://github.com/snacktavish/TreeToReads
cd TreeToReads
conda env create -f environment.yml
conda activate TTR_env
git clone https://github.com/snacktavish/TreeToReads
cd TreeToReads
python setup.py install
(This step is not necessary if you installed TTR as a module)
pip install dendropy
Install seq-gen, software to simulate mutations (http://tree.bio.ed.ac.uk/software/seqgen/)
on ubuntu using apt-get:
sudo apt-get install seq-gen
on Mac or linux (using homebrew, http://brew.sh/):
brew install seq-gen
Install ART, software to generate short reads from simulated genomes (http://www.niehs.nih.gov/research/resources/software/biostatistics/art/)
on ubuntu:
sudo apt install art-nextgen-simulation-tools
on Mac (using homebrew):
brew install art
or wget http://www.niehs.nih.gov/research/resources/assets/docs/artbingreatsmokymountains041716macos64tgz.tgz tar -xzvf artbingreatsmokymountains041716macos64tgz.tgz
add art_illumina to path (see https://coolestguidesontheplanet.com/add-shell-path-osx/)
Install from indelible webpage, and add to path. http://abacus.gene.ucl.ac.uk/software/indelible/
Note: to compile indelible from source for linux installations currently requires adding this include statement to the top of MersenneTwister.h
#include <unistd.h>
(I've reported the issue and it shoud be fixed soon) add indelible to your path
git clone https://github.com/snacktavish/TreeToReads.git
cd TreeToReads
python treetoreads.py example.config
Edit the configuration file, example.config, to fit your data.
Currently generates paired end illumina data. Alternatively, genomes can be generated, and ART may be run separately using any chosen parameters.
The script print out the parameter values and some other useful info. If it runs successfully it will end with "TreeToReads completed successfully!"
The output files will be in the the output directory specified in the config file, e.g. example_out and will consist of:
- fasta_files: A folder containing the simulated genomes for each tip in the tree
- If INDEL simulation is requested,
*_indel.fastafiles will contain the simulated INDELs, with deletions marked by dash (-) characters
- If INDEL simulation is requested,
- fastq: A folder containing folders with the names of each tip from the simulation tree, in each of these folders is the gziped simulated fastq.
- sim.vcf: A vcf file for the simulated mutations with respect to the anchor genome. CURRENTLY DUE TO A BUG VCF FILES ARE NOT GENERATED WHEN INDELS ARE SIMULATED.
- mutsites.txt: An unordered list of the locations of mutations in the genome
- analysis_configuration.cfg: a copy of the control file used for the analysis
- seqgen.out: output messages form the seq-gen software
- simtree.tre.bu: a backup copy of the tree
- simtree.tre: the tree used for simulations: reformatted and polytomies randomly resolved with 0 length branches
- analysis.sh: the bash commands run by the analysis
- seqs_sim.txt: an intermediate file used for generating variable sites
- SNPmatrix: a file in format SEQUENCE, BASE, POSITION describing all variable sites in the genome art_log - log messages from ART software
TreeToReads is also available as a Docker container:
docker pull snacktavish/treetoreads
docker run snacktavish/treetoreads example.config
to run the default example, or
docker run -v /an/example/path:/a/container/path snacktavish/treetoreads /a/container/path/my_treetoreads_config.cfg
to run on real data, where /an/example/path/ contains the file my_treetoreads_config.cfg.
(See the Docker manual for more information about mounting host directories in the container.)
This tool relies on Dendropy, ART, and Seqgen. Please cite them (as well as this repo) in any published work using this simulation pipeline (appropriate citations below)
McTavish E. J., Timme R, (2015) Tree To Reads. https://github.com/snacktavish/TreeToReads bioRxiv
Huang W., Li L, Myers J. R., Marth G. T. (2012). ART: a next-generation sequencing read simulator, Bioinformatics 28 (4): 593-594
Rambaut A. and Grassly N. C. (1997) Seq-Gen: An application for the Monte Carlo simulation of DNA sequence evolution along phylogenetic trees. Comput. Appl. Biosci. 13: 235-238
Sukumaran, J. and Mark T. Holder. 2010. DendroPy: A Python library for phylogenetic computing. Bioinformatics 26: 1569-1571.