Running time benchmark of Tebako generated packed-mn

[ronaldtse]

Using a locally-compiled packed-mn using Tebako on Linux:

root@550f7490e046:/packed-mn-ro# make
...
root@550f7490e046:/packed-mn-ro# time bash -c 'build/bin/metanorma-linux-x86_64 help'
Commands:
  metanorma collection FILENAME -w, --output-folder=OUTPUT_FOLDER  # Render HTML pages from XML/YAML colection
  metanorma compile FILENAME                                       # Compile to a metanorma document
  metanorma config                                                 # Manage configuration file
  metanorma convert FILENAME                                       # Convert STS XML to Metanorma adoc
  metanorma help [COMMAND]                                         # Describe available commands or one specific command
  metanorma list-doctypes                                          # List supported doctypes
  metanorma list-extensions                                        # List supported extensions
  metanorma new NAME -d, --doctype=DOCTYPE -t, --type=TYPE         # Create new Metanorma document
  metanorma site                                                   # Manage site for metanorma collections
  metanorma template-repo                                          # Manage metanorma templates repository
  metanorma version                                                # Version of the code

Options:
  -s, [--no-progress], [--no-no-progress]  # Don't show progress for long running tasks (like download)
                                           # Default: true


real	0m25.262s
user	0m24.899s
sys	0m0.338s

I downloaded the latest ruby-packer generated metanorma 1.5.4pre from here:
https://github.com/metanorma/packed-mn/releases/tag/v1.5.4pre

Results:

root@550f7490e046:/packed-mn-ro# wget https://github.com/metanorma/packed-mn/releases/download/v1.5.4pre/metanorma-linux-x64.tgz
root@550f7490e046:/packed-mn-ro# tar -zxvf metanorma-linux-x64.tgz 
metanorma-linux-x64
root@550f7490e046:/packed-mn-ro# time bash -c './metanorma-linux-x64 help'           
Commands:
  metanorma collection FILENAME -w, --output-folder=OUTPUT_FOLDER  # Render HTML pages from XML/YAML colection
  metanorma compile FILENAME                                       # Compile to a metanorma document
  metanorma config                                                 # Manage configuration file
  metanorma convert FILENAME                                       # Convert STS XML to Metanorma adoc
  metanorma help [COMMAND]                                         # Describe available commands or one specific command
  metanorma list-doctypes                                          # List supported doctypes
  metanorma list-extensions                                        # List supported extensions
  metanorma new NAME -d, --doctype=DOCTYPE -t, --type=TYPE         # Create new Metanorma document
  metanorma site                                                   # Manage site for metanorma collections
  metanorma template-repo                                          # Manage metanorma templates repository
  metanorma version                                                # Version of the code

Options:
  -s, [--no-progress], [--no-no-progress]  # Don't show progress for long running tasks (like download)
                                           # Default: true


real	1m4.038s
user	1m0.904s
sys	0m0.362s

When running the metanorma bundle directly (in the same container):

root@550f7490e046:/packed-mn-ro# time bash -c 'bundle exec metanorma help'
Commands:
  metanorma collection FILENAME -w, --output-folder=OUTPUT_FOLDER  # Render HTML pages from XML/YAML colection
  metanorma compile FILENAME                                       # Compile to a metanorma document
  metanorma config                                                 # Manage configuration file
  metanorma convert FILENAME                                       # Convert STS XML to Metanorma adoc
  metanorma help [COMMAND]                                         # Describe available commands or one specific command
  metanorma list-doctypes                                          # List supported doctypes
  metanorma list-extensions                                        # List supported extensions
  metanorma new NAME -d, --doctype=DOCTYPE -t, --type=TYPE         # Create new Metanorma document
  metanorma site                                                   # Manage site for metanorma collections
  metanorma template-repo                                          # Manage metanorma templates repository
  metanorma version                                                # Version of the code

Options:
  -s, [--no-progress], [--no-no-progress]  # Don't show progress for long running tasks (like download)
                                           # Default: true


real	0m9.250s
user	0m8.102s
sys	0m0.902s

TLDR:

type	real (in s)	user	sys
Native (Ruby 2.7.0)	9.250	8.102	0.902
ruby-packer (Ruby 2.6.3)	64.038	60.904	0.362
Tebako (Ruby 2.7.4)	25.262	24.899	0.338

• Other than the question of why my docker setup seems slow, Tebako provides a real speedup compared to ruby-packer.
• There is still a significant overhead over native (bundle exec). Tebako should have its own benchmarks to see how to optimise.

[maxirmx]

I believe that it is not possible to avoid overhead for simple command like './metanorma-linux-x64 help'
Tebako memfs requires decompression and executable files are extracted to temp

I think for metanorma site generate the difference with native invokation wil be smaller

[ronaldtse]

@maxirmx Why do we need to extract executable files to temp? That's what we're trying to avoid by directly mounting the DwarFS volume? Ruby-packer does not need to extract anything to temp.

[maxirmx]

@ronaldtse
ruby-packer extracts shared libraries to tmp. At least those libraries that it can detect.
Those libraries that ruby-packer cannot detect are extracted by the code added by @CAMOBAP

I do exactly the same. There is no way to load shared libraries, jars etc, from memfs unless this memfs is mounted as conventional file system.

I believe we have discussed it here: tamatebako/tebako#45 (comment)

[ronaldtse]

There is no way to load shared libraries, jars etc, from memfs unless this memfs is mounted as conventional file system.

From glibc, can't we redirect dlopen and friends?

[ronaldtse]

dlopen redirection aside, if we need to extract anything, we should have a consistent extraction path and handle caching (hashing).

And overriding dlopen can be done as shown in https://github.com/riboseinc/retrace:
https://github.com/riboseinc/retrace/blob/70800979fb7fa622fd2c03788b3e06058364ad24/src/v1/dlopen.c

[maxirmx]

From glibc, can't we redirect dlopen and friends?

The issue not with redirecting dlopen, we redirect it already in order to use temp file we create.
dlopen is an entry to libdl (dynamic loader) that opens file and reads it.

You are probably talking not about overriding dlopen but about patching its internals or developing our own dynamic loader. If you mean it, I can create our version of dynamic loader. I will ask for 3-4 months.

[ronaldtse]

You are probably talking not about overriding dlopen but about patching its internals or developing our own dynamic loader.

I was thinking if we can have dlopen that redirects to files loaded via libdwarfs (from dwarfs image directly, not extracted as a temp file). i.e.:

• The shared libraries are provided in the DwarFS image
• We have libdwarfs
• When someone tries to load a shared library via dlopen, we can redirect it to the shared library using a __memfs__ path which is loaded as a DwarFS image without extracting the file.