The dawn of a new port

[elBoberido]

While thinking about the proposals for the service discovery in #784 I came to the conclusion that we are missing a building block.

Let's assume your application has a status/configuration which doesn't change often or even never changes and has a size of only a few kilobyte at maximum if it changes. Let's also assume you want all other applications be able to read this status and their only interest is the latest status. They do not care if there were status changes since they took the previous update. Currently you need a publisher for the application broadcasting this status and a subscriber for each application which is interested in this data. Furthermore, the publisher can serve quite a lot of subscriber, although it's rarely needed. Actually only for situation were an application wants to share a status/config with many applications. Since we use shared memory, we pay a price with a high memory consumption although it's rarely needed and when it's needed, it doesn't even fulfills the job since RouDi can manage more applications than a publisher can serve. It's like when your only tool is a hammer, every problem looks like a nail.

What we need is a StatusPort/ConfigPort/BroadcastPort. Subsequently I will refer to it as StatusPort. This port has the following properties:

• it's never or rarely updated
  • if it's never updated, the data it shares can be large
  • if it's rarely updated, the data it shares should be small
• there are many clients which are interested in the data
• it can have persistent data
  • an application can start, request a StatusPort (with a persistent property), set the data and then stop
  • all applications can still access this data since RouDi has the ownership and therefore the lifetime ends when RouDi stops
• client applications don't access the data directly but by a lambda
  • this prevents torn reads, since the accessor class detects if the data changed during a read operation and re-executes the lambda

While this is not zero copy, when the data is small, it's even faster than the zero copy approach with publisher and subscriber, saves memory and is accessible by all applications.

Potential applications:

• a static config which is shared with all applications
• the service discovery
• the introspection
• the heartbeat
  • currently sent by the unix domain socket ... every 300ms from every app -> unnecessary RouDi wake ups and context switches

With the StatusPort we could reduce the number of subscribers a publisher can serve to e.g. 42 (currently 256). This would result in smaller ports and less cache trashing, which has the potential to even speed things up for publish-subscribe. If it's really necessary to have publish-subscribe with more than these number of ports, we could refactor the publisher ports a little bit. Instead of having the ChunkDistributor as part of the ChunkSender, a ChunkSender could have a vector of pointer to ChunkDistributors. The ChunkDistributors would live in a pool and once the ChunkDistributor cannot hold another ChunkQueue, a ChunkDistributor will be obtained from the pool and added to the vector when a new subscriber is acquired. While this would also solve some problems a StatusPort would solve, a StatusPort still has advantages compared to publish-subscribe for some use cases.

A StatusPort would basically look like this

StatusPortHeader {
    using uint64_t INVALID {0};
    using uint64_t UPDATING {-1};
    std::atomic<uint64_t> generation{INVALID};
}
template <typename T>
using StatusPortData = Sample<T, StatusPortHeader>; // pretty neat, eh? We use a chunk to store the StatusPortData -> this means the number of ports and data sizes can be configured by the user without recompiling RouDi; could potentially also be done for the other ports

template <typename T>
StatusPortWriter {
  public:
    void write(cxx::function_ref<void(T&)> f);
  private:
    StatusPortData<T> data; // potentially also just a `ChunkHeader*`
};

template <typename T>
StatusPortReader {
  public:
    void read(cxx::function_ref<void(const T&)> f);
  private:
    StatusPortData<T> data;
};

If an application has a static configuration to share (no matter how large it is) or needs to update a status to many applications (let's say < 1kB), the StatusPort is much faster than a Publisher/Subscriber combination.

@budrus @mossmaurice @MatthiasKillat @elfenpiff what do you think?

Edit: fixed some findings from MatthiasKillat in the example code

[elBoberido]

Oh, by the way. We could also use a chunk with a user-header to implement a dynamically growing vector or string. The user-header would contain all the methods, a relocatable pointer to the payload and a pointer to a mepoo::MemoryManager. Voilà, dynamic container ;)

[elfenpiff]

@elBoberido I love it!

[MatthiasKillat]

The data it exchanges would need to be trivially copyable I suppose, otherwise you get the problem of a copy or move ctor working on data that is currently updated (see below).

I also think StatusPortData is missing its template argument T in the reader and writer, but this is inconsequential.

Correct me if I am wrong, but I think it would work as follows.
1.) The writer sets the current state (data) via write. BTW: Why do we need/want a function_ref here (also its naming is off)? Probably to increase user flexibility on how to operate on data and also to hide the header from the user. Maybe add a method which is less flexible and only for setting data, the basic case.
2.) The reader accesses this data via lambda (again, the naming is off). Here I sense some problem if the user gets full flexibility she might e.g. invoke a copy ctor but it could work on data that is currently updated. This will break in general and we may have to restrict this. In the basic case the reader copies this data into a temporary structure, checks whether the original changed in the meantime (via StatusPortHeader), and if not copies it to its own datamember, completing the state update.
3) If the update fails, it is probably tried again, until it succeeds (or a maximum time/number of retries is reached). As many lock-free algorithms this has the potential of starvation in theory, but in practice should be ok, especially considering the write updates happen rarely (maybe in bursts).

So far it satisfies what I meant with the state update pattern in #784, but there is one caveat: it does not allow incremental/partial updates of the state, we always copy the whole state. This can probably be solved by composing such ports, one for the full state, one for incremental updates and managing the full state update smartly.

Now it is clear that the update itself can be triggered by whatever method one desires, such as periodic or by being signaled with a condition variable/semaphore or similar.

Edit: partial updates could be handled in the lambda, if information on how to perform the partial update (e.g. what state part did change) is communicated by other means, such as a queue, another StatusPort with different data type (e.g. small queue) etc.

Summary

This all avoids the memory problems to be sure, but care must be taken with the lock-free concurrent copying.
Insofar this approach is superior to all alternatives discussed until now and could be a basic building block, as you say (the name is maybe not final but I think it fits and have no counter proposal at the moment).

[elBoberido]

1. it cannot directly write the data since generation must be updated, therefore the lambda to be able to only partially update the data
2. the read method would take care of that an re-execute the lambda if the generation changes during the current execution. So it does not have to copy the full data but just update it's local copy
3. yes, that's correct

Partial updates are not supported by this port since you cannot know if you slip an update. That needs to be done by publish-subscribe. It could be modeled by the data structure though. Assuming the data structure is an array, you could have a second array with corresponding generation counter. Now you only need to compare this generation counter to a local copy and update only the changed elements.

[MatthiasKillat]

Yes I know, generation is our ABA counter. I would exclude partial updates for now as it can be modeled with this StatusPort or an extension later. It would also get more complicated but not massively so I think. You know, I like optimal solutions to a well defined-problem - and there is still some potential for optimization.

[mossmaurice]

Nice write-up @elBoberido thanks! You've convinced me to use the StatusPorts for #415 :-) See here.

One other idea:
Parameters or static config of whatever sorts are usually available at compile time. So we could in theory also further optimise this use-case by omitting the StatusPortWriter completely and creating the data structure in-place via constexpr. We would need to figure out a interface to the memory provider and this would probably involve a lot of refactoring, so just some food for thought ;)

[mossmaurice]

@elBoberido IIRC we had a discussion about a month ago on whether to use the new StatusPort's also for the introspection. IMHO the ServiceRegistry is a subset of the information which the introspection is displaying. My vision is that we'll transition the introspection topics also to StatusPort. This will have two advantages:

1. Make the usage in ROS 2 easier. Currently there is a method that reads from the introspection topics directly. With iceoryx 2.0 I would switch to the new user-facing DiscoveryInfo API. However, it will not be enough and my feeling is we'll need other user-facing APIs that let's the user introspect the systems. E.g. used by rqt_graph
2. With the introspection topics using our normal pub/sub classes potentially a rogue user could crash RouDi if it keeps putting his fingers on the chunks without releasing them. This won't be possible with the StatusPort's

What's your take on the idea?

cc @budrus

[elBoberido]

@mossmaurice I think the introspection would be a good candidate for the status ports