Race Conditions in qutil

[insertinterestingnamehere]

I'm seeing a handful of thread sanitizer errors about race conditions in the qutil code.

One example: apparently the macro invoked at

qthreads/src/qutil.c

Line 152 in a2d1dba

INNER_LOOP(qutil_uint_mult_inner, qutil_uis_args, MULT_MACRO)

,

qthreads/src/qutil.c

Line 156 in a2d1dba

INNER_LOOP(qutil_uint_max_inner, qutil_uis_args, MAX_MACRO)

, and

qthreads/src/qutil.c

Line 160 in a2d1dba

INNER_LOOP(qutil_uint_min_inner, qutil_uis_args, MIN_MACRO)

includes both a read and a corresponding atomic write from another thread.

Another example:
non-atomic read:

qthreads/src/qutil.c

Line 897 in a2d1dba

aligned_t cur = *args->furthest_leftwall;

atomic write to the same variable:

qthreads/src/qutil.c

Line 903 in a2d1dba

tmp = qthread_cas(args->furthest_leftwall, cur, mine);

Similar:

qthreads/src/qutil.c

Line 909 in a2d1dba

aligned_t cur = *args->furthest_rightwall;

qthreads/src/qutil.c

Line 915 in a2d1dba

tmp = qthread_cas(args->furthest_rightwall, cur, mine);

Another more unusual one:
The macro invoked at

qthreads/src/qutil.c

Line 170 in a2d1dba

INNER_LOOP(qutil_int_mult_inner, qutil_is_args, MULT_MACRO)

has a non-atomic write as well as an atomic write to the same variable from a different thread via qthread_syncvar_fill at

qthreads/src/syncvar.c

Line 609 in a2d1dba

UNLOCK_THIS_MODIFIED_SYNCVAR(addr, ret, 0);

.

[insertinterestingnamehere]

Poking at some of the memory fences for syncvars seems like a partial fix. Some additional atomics in the nemesis scheduler may be necessary to get the sequencing right, but at this point I'm still not sure. The weird thing though is that some of the thread sanitizer tracebacks here seem to be getting garbled and it's not at all clear why/how that's happening. It may be that a better-debugged #249 is needed to fix these warnings.

[insertinterestingnamehere]

Update on this one, I'm fairly sure a lot (all?) of these errors are caused by thread sanitizer not being able to properly follow dependencies through the atomic logic in our various threadqueues. It's fixable, but it'll require an extension beyond the interop in #249.