Dual NI + JVM Mode for Loading Libraries

[JaroslavTulach]

• since Handle --jvm flag in process with JNI instantiated JVM #12843
• there is a way to load JVM inside of the same process as NI
• there is an effort to make enso executable smaller - Make native `enso` launcher smaller! #12446
• by small steps: Remove embedded native libraries from JARs #13150
• but what we need is:

Dual Execution Mode

Design a system where we can load a class from NI (as is done regularly since 2025.1 release) as well as from the embedded HotSpot JVM (as introduced by #12843). We need:

• Sending interop messages via byte[] boundary between two JVMs
  • probably reuse Persistance to serde - Channel connecting two JVM instances #13206
  • need a separate/own persistance pool independent from the existing ones - done in Persistable.Pool to group related instances into pool #13201
  • deal with two memory arenas
• Remote Object references in two GCs needed - outline of the work
  • deal with two garbage collectors in two JVMs (e.g. SVM and HotSpot VM)
• System to launch a JVM or ClassLoader or ModuleLayer per a library
  • relates to Package libraries Java code as Java Modules #10714
  • and also Register services in package.yaml instead via ServiceLoader #11868

When we have this system, we can avoid compiling every Standard. library into NI and leave those less likely to use to be loaded by JVM only.

[Akirathan]

Sending interop messages between two JVMs seems reasonable. Is there any other way how these two JVMs can send each other objects without the need to serialize them? Can they share a region of memory?

[JaroslavTulach]

Sending interop messages between two JVMs seems reasonable. Is there any other way how these two JVMs can send each other objects without the need to serialize them? Can they share a region of memory?

• primitive values (e.g. valid protocol values will be serialized and send by a copy
• the TruffleObject instances will be sent by reference
• prior work can be found in AbstractFXPresenter for example
  • the presenter bridges two VMs: a JVM and a JavaScript VM
  • when sending a value toJavaScript
    • primitive values are send as they are - e.g. by copy
    • arrays get a special treatment and are sent by copy - we don't need that array is just a TruffleObject
    • all other values (e.g. TruffleObject in our case) are handled by JavaValues map
    • they get an ID and only a reference to that ID is sent to the other VM
  • when receiving the value back toJava
    • the ID is extracted
    • appropriate TruffleObject is found in the map by its ID
    • the object is then passed around in JVM until it is send back to the other VM when the existing ID is reused as a reference
  • some care needs to be taken to use java.lang.ref.Reference to clean up no longer needed objects
    • however each VM has its own GC
    • should there be a cycle of references crossing back and forth the VMs, it never garbage collects
• another version of the prior work is in the Generic presenter
• when implementing this on the JVM one could use JNI references - but I probably stick to plain ID hash map to begin with (easier to test and debug) and only consider using JNI later

[JaroslavTulach]

Currently rejected idea: reuse RMI

• appropriate TruffleObject is found in the map by its ID
• the object is then passed around in JVM until it is send back to the other VM when the existing ID is reused as a reference
• however each VM has its own GC

Pros:

• working implementation
• provides implementation of DGC - e.g. distributed GC
  • we will have to mimic it

Cons:

• relies on java.io.ObjectOutputStream - e.g. serialization
  • serialization isn't first class citizen in native image
  • serialization is known to be slow
• unsure how to reuse RMI without sockets
  • we want to transfer data via MemorySegment
  • how easy it is hook into RMI and use it for slightly different purpose?

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2025-06-02):

Progress: .

• Exchange messages between JVMs: 5b42e1d
• reuse RMI rejected: Dual NI + JVM Mode for Loading Libraries #13172 (comment)
• various meetings

. It should be finished by 2025-06-10.