Service: improve support for other async runtimes

[cecton]

Use case

Ideally we would like the user to be able use the async runtime they want. This is more or less possible but our current implementation doesn't allow us to use async-std.

Problem encountered

I tried to use async-std instead of tokio. Even though it works, it produces unexpected behavior as the node's tasks are not gracefully shutdown when the main task completes. Which is a problem because some objects are not dropped before the process exit.

I first thought it was a bug but it is an intentional behavior of async-std to be closer to std::thread. See async-rs/async-std#832 for more details.

Solutions

First solution

One solution would be to support the 2 different patterns:

• in tokio, the tasks are canceled and awaited when the tokio runtime is dropped, nothing needs to be done manually
• in async-std, to do a graceful exit, the tasks need to be cancelled and awaited manually

To do that we need:

1. a custom JoinHandle trait that would have a fn cancel() and impl Future (to await the end of the task)
2. when fn terminate is called on the task manager, all the JoinHandle will be called with fn cancel()
3. when using the tokio runtime we would use a dummy JoinHandle that does nothing on cancel
4. when using async-std runtime we would use a JoinHandle that calls the underlying cancellation method of async-std's JoinHandle

Alternative solution

Right now the TaskExecutor uses a function to spawn a task that doesn't return anything. Another solution would be for the TaskExecutor to use a function that would return a cancellation function. The TaskManager would need to keep a list of all the cancellation functions gathered during the execution of the node and call them one by one (or in parallel with join!) when exiting.

This solution would avoid introducing a new trait.

Concerns

I don't know how many background tasks we spawn or even if this number is static. We might just fill up the memory with JoinHandles or cancellation functions...

cc @expenses @gnunicorn @tomaka