new indexing rules in DataFrames v0.19 (#102)

* code:
  - update to new indexing rules in DataFrames v0.19
  - getIndex -> findfirst
  - readNexusTrees: more efficient
* documentation:
  - discrete trait page
  - Documenter v0.23
* tests:
  - folder clean-up
  - tests added to increase coverage
  - removed match_mode=:any in log tests

.travis.yml

@@ -6,6 +6,7 @@ os:
   - osx
 julia:
   - 1.1
+  - 1.2
 
 notifications:
   email: false
@@ -24,7 +25,7 @@ coveralls: true
 jobs:
   include:
     - stage: "Documentation"
-      julia: 1.0
+      julia: 1.2
       os: linux
       script:
         - julia --project=docs/ -e 'using Pkg; Pkg.instantiate();
@@ -35,6 +36,6 @@ jobs:
         - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9; fi
         - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo add-apt-repository -y "deb http://cran.rstudio.com/bin/linux/ubuntu $(lsb_release -s -c)/"; fi
         - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo apt-get update -qq -y; fi
-        - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo apt-get install git r-base r-base-dev r-recommended -y; fi
+        - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo apt-get install --allow-unauthenticated git r-base r-base-dev r-recommended -y; fi
       # - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo R -e 'install.packages("ggplot2", dep=TRUE, repos="http://cran.us.r-project.org")'; fi # takes ~ 6min 2018-06-02
       after_success: skip

docs/Project.toml

@@ -13,4 +13,4 @@ StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
 StatsModels = "3eaba693-59b7-5ba5-a881-562e759f1c8d"
 
 [compat]
-Documenter = "~0.22"
+Documenter = "~0.23"

docs/make.jl

@@ -4,6 +4,7 @@ using Pkg
 Pkg.add(PackageSpec(name="PhyloPlots", rev="master"))
 
 using PhyloNetworks
+DocMeta.setdocmeta!(PhyloNetworks, :DocTestSetup, :(using PhyloNetworks); recursive=true)
 
 makedocs(
     sitename = "PhyloNetworks.jl",

docs/readme.md

@@ -100,9 +100,10 @@ or interactively in `docs/`:
 pkg> activate .
 pkg> status # just to check
 pkg> status --manifest
-pkg> instantiate
-pkg> # dev PhyloPlots # to get the master branch
-pkg> dev ~/.julia/dev/PhyloNetworks
+pkg> instantiate # after deleting Manifest.toml and undo changes to Project.toml
+pkg> # add RCall#master # in case some dependency causes an issue
+pkg> dev PhyloNetworks
+pkg> add PhyloPlots#master # to get the master branch: done by make.jl
 julia> include("make.jl")
 ```
 

docs/src/index.md

@@ -10,7 +10,7 @@ and their use for trait evolution.
 **How to get help**
 
 - the package [wiki](https://github.com/crsl4/PhyloNetworks.jl/wiki) has a step-by-step
-  tutorial, done for the 2018 MBL workshop, with background on networks and
+  tutorial, done for the 2019 MBL workshop, with background on networks and
   explanations.
 - the [google group](https://groups.google.com/forum/#!forum/phylonetworks-users)
   has answers to common questions.
@@ -28,7 +28,7 @@ and their use for trait evolution.
   Phylogenetic Comparative Methods for Phylogenetic Networks with Reticulations.
   Systematic Biology, 67(5):800–820.
   [doi:10.1093/sysbio/syy033](https://doi.org/10.1093/sysbio/syy033).
-- Claudia Solís-Lemus and Cécile Ané(2016).
+- Claudia Solís-Lemus and Cécile Ané (2016).
   Inferring Phylogenetic Networks with Maximum Pseudolikelihood under Incomplete Lineage Sorting.
   [PLoS Genet](http://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1005896)
   12(3):e1005896. [doi:10.1371/journal.pgen.1005896](https://doi.org/10.1371/journal.pgen.1005896)

docs/src/lib/internals.md

@@ -1,8 +1,5 @@
 ```@meta
 CurrentModule = PhyloNetworks
-DocTestSetup = quote
-  using PhyloNetworks
-end
 ```
 
 # Internal Documentation
@@ -34,8 +31,3 @@ Modules = [PhyloNetworks]
 Public = false
 Order   = [:function, :constant]
 ```
-
-```@meta
-DocTestSetup = nothing
-```
-

docs/src/lib/public.md

@@ -4,13 +4,6 @@ Documentation for `PhyloNetworks`'s public (exported) interface.
 
 See [Internal Documentation](@ref) for documentation on internal functions.
 
-```@meta
-DocTestSetup = quote
-    using PhyloNetworks
-end
-DocTestFilters = r" PhyloNetworks .*:\d+"
-```
-
 ```@contents
 Pages = ["public.md"]
 ```
@@ -65,6 +58,7 @@ readTopology
 readTopologyLevel1
 readInputTrees
 readMultiTopology
+readNexusTrees
 readSnaqNetwork
 readTrees2CF
 readTableCF
@@ -143,8 +137,3 @@ nstates
 stationary
 empiricalDNAfrequencies
 ```
-
-```@meta
-DocTestSetup = nothing
-DocTestFilters = nothing
-```

docs/src/man/bootstrap.md

@@ -137,7 +137,7 @@ which tree edges have bootstrap support lower than 100% (none here) with
 ```@repl bootstrap
 using DataFrames # for showall() below
 show(BSe_tree, allrows=true, allcols=true)
-BSe_tree[BSe_tree[:proportion] .< 100.0, :]
+filter(row -> row[:proportion] < 100, BSe_tree)
 ```
 Finally, we can map the bootstrap proportions onto the network or its main tree
 by passing the bootstrap table to the `edgeLabel` option of `plot`:
@@ -163,7 +163,7 @@ output file of `snaq!` earlier, for consistency across different Julia sessions.
 If we wanted to plot only certain bootstrap values, like those below 100% (1.0),
 we could do this:
 ```julia
-plot(net1, :R, edgeLabel=BSe_tree[BSe_tree[:proportion] .< 100.0, :]);
+plot(net1, :R, edgeLabel=filter(row -> row[:proportion] < 100, BSe_tree));
 ```
 
 ## support for hybrid edges and hybrid nodes
@@ -204,7 +204,8 @@ To see what is the clade named "H7", for instance:
 ```@repl bootstrap
 BSc # this might be too big
 show(BSc, allrows=true, allcols=true)
-BSc[:taxa][BSc[:H7]]
+# BSc[BSc[!,:H7], :taxa] # just a different syntax to subset the data in the same way
+filter(row -> row[:H7], BSc).taxa
 ```
 We can also get bootstrap values associated with edges, to describe the support that a given
 hybrid clade has a given sister clade.
@@ -225,7 +226,7 @@ We can plot the bootstrap values of the 2 hybrid edges in the best network:
 ```@example bootstrap
 R"svg(name('boot_net_net.svg'), width=4, height=4)" # hide
 R"par"(mar=[0,0,0,0]) # hide
-plot(net1, :R, edgeLabel=BSe[[:edge,:BS_hybrid_edge]]);
+plot(net1, :R, edgeLabel=BSe[!,[:edge,:BS_hybrid_edge]]);
 R"dev.off()" # hide
 nothing # hide
 ```
@@ -240,7 +241,7 @@ of bootstrap networks. In another 1% bootstrap, A received gene flow from anothe
 ```@example bootstrap
 R"svg(name('boot_net_ret.svg'), width=4, height=4)" # hide
 R"par"(mar=[0,0,0,0]) # hide
-plot(net1, :R, nodeLabel=BSn[[:hybridnode,:BS_hybrid_samesisters]]);
+plot(net1, :R, nodeLabel=BSn[!,[:hybridnode,:BS_hybrid_samesisters]]);
 R"dev.off()" # hide
 nothing # hide
 ```
@@ -250,11 +251,12 @@ Below is example code to place tree edge support and hybrid edge support
 on the same plot.
 
 ```julia
-tmp = BSe[!isna(BSe[:edge]),[:edge,:BS_hybrid_edge]]
-rename!(tmp, :BS_hybrid_edge, :proportion)
-rename!(tmp, :edge, :edgeNumber)
+tmp = filter(row -> !ismissing(row[:edge]), BSe) # filter rows
+select!(tmp, [:edge,:BS_hybrid_edge])            # select 2 columns only
+rename!(tmp, :BS_hybrid_edge => :proportion)     # rename those columns, to match names in BSe_tree
+rename!(tmp, :edge => :edgeNumber)
 tmp = vcat(BSe_tree, tmp)
-plot(net1, edgeLabel=tmp, nodeLabel=BSn[[:hybridnode,:BS_hybrid_samesisters]])
+plot(net1, edgeLabel=tmp, nodeLabel=BSn[!, [:hybridnode,:BS_hybrid_samesisters]])
 ```
 
 ### Who are the hybrids in bootstrap networks?
@@ -268,12 +270,12 @@ being of hybrid origin.
 ```@example bootstrap
 R"svg(name('boot_net_hyb_1.svg'), width=4, height=4)" # hide
 R"par"(mar=[0,0,0,0]) # hide
-plot(net1, :R, nodeLabel=BSn[BSn[:BS_hybrid].>0, [:hybridnode,:BS_hybrid]]);
+plot(net1, :R, nodeLabel=filter(row->row[:BS_hybrid]>0, BSn)[!,[:hybridnode,:BS_hybrid]]);
 R"dev.off()" # hide
 nothing # hide
 R"svg(name('boot_net_hyb_2.svg'), width=4, height=4)" # hide
 R"par"(mar=[0,0,0,0]) # hide
-plot(net1, :R, edgeLabel=BSn[BSn[:BS_hybrid].>0, [:edge,:BS_hybrid]]);
+plot(net1, :R, edgeLabel=filter(row->row[:BS_hybrid]>0, BSn)[!,[:edge,:BS_hybrid]]);
 R"dev.off()" # hide
 nothing # hide
 ```
@@ -290,12 +292,12 @@ We filtered clades to show those with sister support > 5%:
 ```@example bootstrap
 R"svg(name('boot_net_clade_1.svg'), width=4, height=4)" # hide
 R"par"(mar=[0,0,0,0]) # hide
-plot(net1, :R, nodeLabel=BSn[BSn[:BS_minor_sister].>5, [:node,:BS_minor_sister]]);
+plot(net1, :R, nodeLabel=filter(r->r[:BS_minor_sister]>5, BSn)[!,[:node,:BS_minor_sister]]);
 R"dev.off()" # hide
 nothing # hide
 R"svg(name('boot_net_clade_2.svg'), width=4, height=4)" # hide
 R"par"(mar=[0,0,0,0]) # hide
-plot(net1, :R, edgeLabel=BSn[BSn[:BS_minor_sister].>5, [:edge,:BS_minor_sister]]);
+plot(net1, :R, edgeLabel=filter(r->r[:BS_minor_sister]>5, BSn)[!,[:edge,:BS_minor_sister]]);
 R"dev.off()" # hide
 nothing # hide
 ```
@@ -311,7 +313,7 @@ but there is much uncertainty about its exact placement and about its direction.
 
 Mapping the support for major sister clades might be interesting too:
 ```julia
-plot(net1, nodeLabel=BSn[BSn[:BS_major_sister].>5, [:node,:BS_major_sister]])
+plot(net1, nodeLabel=filter(r->r[:BS_major_sister]>5, BSn)[!,[:node,:BS_major_sister]])
 ```
 
 The estimated heritability γ on hybrid edges in the reference network, when present in a

docs/src/man/dist_reroot.md

@@ -86,6 +86,7 @@ R"svg(name('reroot_net7taxa_1.svg'), width=4, height=4)" # hide
 R"par"(mar=[0,0,0,0]) # hide
 plot(net7taxa, :R, showGamma=true, showEdgeNumber=true, tipOffset=0.2);
 R"dev.off()"; # hide
+nothing # hide
 ```
 ![reroot net7taxa 1](../assets/figures/reroot_net7taxa_1.svg)
 
@@ -115,6 +116,7 @@ rootonedge!(net7taxa, 5);
 plot(net7taxa, :R, showGamma=true, tipOffset=0.2);
 R"mtext"("rooted on hybrid edge 5 (minor)", line=-1);
 R"dev.off()"; # hide
+nothing # hide
 ```
 ![reroot net7taxa 2](../assets/figures/reroot_net7taxa_2.svg)
 
@@ -132,6 +134,7 @@ R"par"(mar=[0,0,0,0]); # hide
 plot(net7taxa, :R, tipOffset=0.2); # not showing gamma values, because we changed them artificially
 R"mtext"("rooted on hybrid edge 5 (considered major)", line=-1);
 R"dev.off()"; # hide
+nothing # hide
 ```
 ![reroot net7taxa 3](../assets/figures/reroot_net7taxa_3.svg)
 

docs/src/man/expectedCFs.md

@@ -27,7 +27,7 @@ Here is one way to plot them, via R again, and using the R package `ggplot2`.
 
 ```@example expCFs
 using RCall
-obsCF = df_long[:obsCF]; expCF = df_long[:expCF]; # hide
+obsCF = df_long[!,:obsCF]; expCF = df_long[!,:expCF]; # hide
 R"name <- function(x) file.path('..', 'assets', 'figures', x)"; # hide
 R"svg(name('expCFs_obsvsfitted.svg'), width=5, height=4)"; # hide
 R"par"(mar=[2.5,2.6,.5,.5], mgp=[1.5,.4,0], tck=-0.01, las=1, pty="s"); # hide
@@ -75,23 +75,26 @@ Many points are overlapping, like before, so they are again "jittered" a bit.
 
 ```@example expCFs
 using DataFrames
-df_long[:has_A] = "no" # add a column to our data, to indicate which 4-taxon sets have A or not
+df_long[!,:has_A] .= "no"; # add a column to our data, to indicate which 4-taxon sets have A or not
 for r in eachrow(df_long)
     if "A" ∈ [r[:tx1], r[:tx2], r[:tx3], r[:tx4]]
        r[:has_A]="yes"
     end
 end
-has_A = df_long[:has_A]; # hide
-nq = length(has_A) # hide
+has_A = df_long.has_A # hide
+nq = length(has_A); # hide
 R"colA=rep('#008080',$nq); bgA=rep('#00808090',$nq);"; # hide
 R"colA[$has_A=='yes']='#F8766D'; bgA[$has_A=='yes']='#F8766D90'"; # hide
 R"svg(name('expCFs_obsvsfitted_A.svg'), width=5, height=4)"; # hide
 R"par"(mar=[2.5,2.6,.5,.5], mgp=[1.5,.4,0], tck=-0.01, las=1, pty="s"); # hide
 R"plot(0:1, 0:1, type='l', bty='L', lwd=0.3, col='black', xlab='quartet CF observed in gene trees', ylab='quartet CF expected from network')"; # hide
-R"set.seed"(2345) # hide
+R"set.seed"(2345); # hide
 R"points(jitter($obsCF,amount=0.005),jitter($expCF,amount=0.005),col=colA,bg=bgA,pch=21)"; # hide
 R"legend(x=0.7,y=0.3,pch=21,col=c('#008080','#F8766D'),legend=c('no','yes'),title='has A?', bty='n',bg=c('#00808090','#F8766D90'))"; # hide
 R"dev.off()"; # hide
+nothing # hide
+```
+```@repl expCFs
 first(df_long, 7) # first 7 rows
 ```