Multiple interpreters

[Hadeweka]

Based on #26, this thread can be used to discuss how exactly to implement support for multiple interpreters.

Some of the problems:

• Multiple interpreters could refer to the same object in Crystal, so each interpreter would need a separate reference table and a way to make it thread-safe
• Since Crystal is able to store references to Ruby values, passing a value between interpreters would likely lead to crashes, so there needs to be a way to prevent that
• Communication between the interpreters in general would require a dedicated solution (e.g. channels passing marshalled data)
• There are two options to inform Anyolite functions of the interpreter to be used: Direct passing of the interpreter as an argument (which would actually make code working with Anyolite much more complicated and would require breaking changes) or a global state (which is mostly the default now, but would require switching to the respective interpreter and staying there in a thread-safe way each time) - both have their advantages and disadvantages
• Each interpreter has some startup time, so creating hundreds of interpreters for a single application would lead to bad performance

Implementing multiple interpreters is definitely something I'd like to do, but all of the problems above need to be addressed, first.

My major question currently is: Are there actual use cases where the advantages outweigh these disadvantages?

[chendo]

My goal is to build a high throughput and (ideally low latency) HTTP server capable of running sets of scripts which can be updated dynamically. The idea is to have a pool of prepared interpreters that each requests checks out from, runs the scripts, and returns the appropriate output. I'm enabling -D preview_mt which obviously introduces some issues but I'm hoping to be able to resolve them.

I'm also trying to achieve this with Duktape.cr at the moment, and have had some minor success, even with preview_mt.

However, there are some issues. Duktape.cr handles pool reuse fine, but calling into Crystal from JS is experimental. If there's a script that calls out to Crystal and causes a Fiber switch (IO, sleep, etc), resumption of the Fiber can cause crashes. It also does not easily facilitate calling Crystal instances (only static methods), which is a little bit annoying.

However, I can achieve 100k rps on my M1X MBP when reusing pools (but not shelling out to IO). Creating a new Duktape::Runtime for every request is slow (4k rps), but this doesn't crash even with IO, and scales alright by increasing CRYSTAL_WORKERS, to 13k rps with 8 workers.

My ideal situation would be able to use the same pool of Anyolite interpreters while being able to do IO. I don't require communication between interpreters, or passing values between interpreters.

[Hadeweka]

The problem for any implementation is that there would still need to be a safeguard for the values, which would make the code for Anyolite much more complicated.

Even if you don't plan to share values, others might want to (or at least attempt to), and I don't want Anyolite to behave unpredictably (which it would, if there'd be no safeguard or dedicated way to share values).