thread safety issues

[Geal]

As a reference, in PoolInner:

• next is the index of the head of a linked list of element (elements that are released after use are pushed onto a lock-free stack). If an element has never been obtained from the pool, its next value points to the element right after
• init is the number of initialized elements
• count is the number of allocated elements (ie for which we actually obtained memory pages)
• maximum is the maximum number of elements the pool could hold (if we used all the address space we reserved)

element initialization is not thread safe

I want to decorrelate element initialization from the pool creation (with a growable pool, it's necessary), but the current code is not safe because the bound of initialized elements self.init is accessed during checkout: https://github.com/sozu-proxy/poule/blob/master/src/lib.rs#L244

Here's the rough process to initialize one element:
1- check that self.init < self.count: the count value indicates how many elements were allocated, we want to know if some of them are still not initialized
2- initialize the element at index self.init and set its next value to self.init + 1
3- self.init += 1

Right now, init is not an atomic, but even if I make its operations atomic, it will not be safe:

• thread A and B do 1 at the same time, and end up initializing the same element at the same time
• to avoid that, we could change the order to 1, 3, 2, but then another thread could call checkout() between 3 and 2, and end up with an uninitialized element

growing available memory is not thread safe either

self.count is not atomic, but growing the memory is mostly idempotent: if you want to increase memory to X in one thread, and Y i another, with X < Y, available memory will be Y

Possible solutions

I see multiple approaches here:

• wrap everything in a mutex (well, obviously)
• growing and allocating at the same time instead of separating them
• see it as multiple queues:
  • the queue of allocated elements: the start of the queue is the end of currently allocated elements, end of the queue indicates memory we're allocating (once allocation is done, consume those)
  • the queue of initialized elements: the start of the queue is the currently initialized elements, the rest of the queue contains elements in the middle of allocation

lock-free programming is really not my strong suit, so I'd appreciate some help on this