Use synchronized to guard state, ReentrantReadWriteLock for blocking calls

[swankjesse]

This is what I wrote in #8290

I think there’s two very different ways we use synchronized:

• To guard in-memory state. For the accesses we acquire the lock, read or write some fields, and release the lock. There’s unlikely to be any > contention for these, and if there is it’s likely to be very short-lived.
• To guard I/O. For these accesses we acquire the lock, perform some potentially-slow I/O operation, and release the lock. These ones are > likely to have contention.

I think we definitely want to change 2 to Loom-safe locks. I am unconvinced that there’s a benefit for changing 1. But there is a potential memory + performance cost.

I’d like to find all of the things we converted to ReentrantReadWriteLock to guard in-memory state, and change ’em back to use synchronized.

This link from @yschimke provides more discussion!

[yschimke]

Let's make sure we have good coverage of Loom tests when we put this in. I think I landed a test that fails if we block loom.

[swankjesse]

Oooh good call.