nextflow conda launcher can pick the wrong java version

[cement-head]

After install, I get this:

(/home/cbfgws6/Programs/nextflow/env/polishCLR_env) cbfgws6@trinity:~/Programs/nextflow$ nextflow run isugifNF/polishCLR -r main --check_software
/home/cbfgws6/Programs/nextflow/env/polishCLR_env/lib/jvm/bin/java: symbol lookup error: /home/cbfgws6/Programs/nextflow/env/polishCLR_env/lib/jvm/bin/java: undefined symbol: JLI_StringDup
NOTE: Nextflow is trying to use the Java VM defined by the following environment variables:
 JAVA_CMD: /home/cbfgws6/Programs/nextflow/env/polishCLR_env/lib/jvm/bin/java
 NXF_OPTS:

Yet, openjdk 11 is installed in the conda environment.

How do I fix this?

[j23414]

Thanks for the message! Could you run the following to see if it returns the same error?

nextflow -version

I have a suspicion that it's related to this issue: nextflow (conda) launcher didn't pick java executable correctly.

Can also try running JAVA_CMD

/home/cbfgws6/Programs/nextflow/env/polishCLR_env/lib/jvm/bin/java --version

It's possible the above symlink needs to be replaced with the openjdk 11 version but we'd have to find where the openjdk-11 bin folder is in the system/conda.

[cement-head]

Okay, here you go:

$ nextflow -version

      N E X T F L O W
      version 22.10.1 build 5828
      created 27-10-2022 16:58 UTC (12:58 EDT)
      cite doi:10.1038/nbt.3820
      http://nextflow.io

But, there's something wierd here...polishCLR doesn't seem to be in an actual conda environment. I followed the installation instructions and now I get this error:

$ conda activate env/polishCLR_env

EnvironmentLocationNotFound: Not a conda environment: /home/cbfgws6/env/polishCLR_env

$ conda info --envs
# conda environments:
#
                         /home/cbfgws6/Programs/nextflow/env/polishCLR_env
base                     /home/cbfgws6/miniconda3
haslr                    /home/cbfgws6/miniconda3/envs/haslr
masurca                  /home/cbfgws6/miniconda3/envs/masurca
pb                       /home/cbfgws6/miniconda3/envs/pb
quast                    /home/cbfgws6/miniconda3/envs/quast
seqkit                   /home/cbfgws6/miniconda3/envs/seqkit
tgs-gapcloser            /home/cbfgws6/miniconda3/envs/tgs-gapcloser

What's going on?

[j23414]

Ah I see what's happening, the polishCLR_env on your system would need to be activated using:

conda activate /home/cbfgws6/Programs/nextflow/env/polishCLR_env

But could you run a which java before and after the above command? Because it seems like your native nextflow -version was working but JAVA_CMD might be reset after activation.

Some Background

On HPC, we're usually placing the polishCLR_env in a shared project folder such that collaborators can reuse the same conda environment (-p ${PWD}/env/polishCLR_env) and submit their own slurm jobs. Otherwise, if we install polishCLR_env using the default install location:

/home/cbfgws6/miniconda3/envs//polishCLR_env

Only the person who installed the environment cbfgws6 can access and activate the conda environment.

Side Note: Creating a regular conda environment would be:

conda env create -f environment.yml

And then conda activate polishCLR_env will work. I'll need to think through how to write better documentation about creating/activating the polishCLR_env environment.

[cement-head]

$ conda activate polishCLR_env
(polishCLR_env) cbfgws6@trinity:~/Programs/nextflow$ nextflow -version
/home/cbfgws6/miniconda3/envs/polishCLR_env/lib/jvm/bin/java: symbol lookup error: /home/cbfgws6/miniconda3/envs/polishCLR_env/lib/jvm/bin/java: undefined symbol: JLI_StringDup
NOTE: Nextflow is trying to use the Java VM defined by the following environment variables:
 JAVA_CMD: /home/cbfgws6/miniconda3/envs/polishCLR_env/lib/jvm/bin/java
 NXF_OPTS:

cbfgws6@trinity:~$ which java
/usr/bin/java
cbfgws6@trinity:~$ conda activate polishCLR_env
(polishCLR_env) cbfgws6@trinity:~$ which java
/home/cbfgws6/miniconda3/envs/polishCLR_env/bin/java
(polishCLR_env) cbfgws6@trinity:~$

[j23414]

Since this seems to be an ongoing issue with Nextflow & conda and I am not a maintainer of Nextflow.

• nexflow (conda) launcher didn't pick java executable correctly under macOS nextflow-io/nextflow#1523

I highly recommend trying the docker or singularity methods instead.

• https://github.com/isugifNF/polishCLR#docker
• https://github.com/isugifNF/polishCLR#singularity