Skip to content

Latest commit

 

History

History
133 lines (96 loc) · 4.48 KB

README.md

File metadata and controls

133 lines (96 loc) · 4.48 KB
Raw

Update spack develop version

🔋 Spack with batteries included (linux/x86_64)

Spack is a package manager, and package managers should be trivial to install.

This repo offers a single, static executable for Spack:

$ wget -qO spack.x https://github.com/eth-cscs/spack-batteries-included/releases/download/develop/spack-x86_64.x
$ chmod +x spack.x
$ ./spack.x install curl tls=mbedtls

What version of Spack is shipped?

The URL above gives you a rolling release of Spack's develop branch, which is updated hourly. The exact commit SHA is included as a file and can be retrieved like this:

$ spack.x --squashfs-extract spack_sha && cat spack/spack_sha
[prints the Spack commit sha]

Supported platforms

  • CentOS 7 and above
  • Ubuntu 14.04 and above
  • Debian 8 and above
  • Fedora 20 and above
  • SUSE Linux 13 and above
  • Arch Linux
  • Gentoo
  • Windows Subsystem for Linux 2 with any of the above distro's.

The system dependencies are glibc 2.17 and above and optionally the fusermount executable. If your system supports rootless containers it likely has fusermount installed already!

How does it work?

spack.x consists of a modified version of the AppImage runtime concatenated with a big squashfs file which includes binutils, bzip2, clingo, curl, file, git, gmake, gpg, gzip, openssl, patch, patchelf, python, py-boto3, tar, unzip, xz, zstd and their dependencies.

When you run spack.x [args] it will use fusermount to mount this squashfs file in a temporary directory, and then execute the entrypoint executable spack.

The spack executable sets some environment variables like PATH and DL_LIBRARY_PATH to the bin and lib folders of the squashfs file, and then it executes python3 spack_src/bin/spack [args].

When the command is done running, the runtime unmounts the squashfs file again.

My system doesn't allow me to use fusermount, what now?

fusermount is used to mount a squashfs file included in the binary. If you don't want that, you can just extract it:

$ spack.x --squashfs-extract
$ ./spack/spack
usage: spack [-hkV] [--color {always,never,auto}] COMMAND ...

but working with the extracted spack folder can come with a performance penalty on shared filesystems in HPC centers.

Differences and improvements over AppImage runtime

  • spack.x uses zstd for faster decompression;
  • spack.x itself is an entirely static binary;
  • spack.x does not need to dlopen libfuse.so.

Troubleshooting

immutability The squashfs mountpoint is a readonly folder, meaning that spack can't write to spack/{var,opt} folders. spack.x is configured to use some non-standard directories, see spack.x config blame config for details.

Note, spack.x applies this patch to ensure that log files are written to the config:misc_cache folder.

openssl: By default spack.x uses ca-certificates-mozilla for downloading package sources over https. If you somehow need to use system certificates, set SSL_CERT_DIR and GIT_SSL_CAINFO or SSL_CERT_FILE and GIT_SSL_CERT.

Can I run spack.x inside a container?

Yes, but please don't! Since fusermount is a setuid binary, you will need to run a privileged container, which is never a good idea.

The recommended way to run spack.x inside a container is to just extract it:

$ spack.x --squashfs-extract
$ ./spack/spack --version

If you insist on running spack.x in Docker, this is one way to do it:

$ sudo docker run --privileged --device /dev/fuse -it -v $PWD/spack.x:/bin/spack.x ubuntu:18.04
# apt update && apt install fuse # install fusermount
# spack.x --version

Running an executable shipped with spack.x directly

If you want to run an executable shipped with spack.x directly instead of invoking spack (the default entrypoint), try this:

$ NO_ENTRYPOINT= spack.x which python
/tmp/.mount_spack.h0zr1h/view/bin/python

How do I build spack.x myself?

Initially you may need docker to get a rootfs filesystem for centos 7.

Building goes like this:

make rootfs-with-spack
make

You'll find the output in

build/output