Experimenting With Cirrus CI

[insertinterestingnamehere]

Experimenting with Cirrus CI since it seems to be an easy way to get builds on both Arm and Mac M1.

[insertinterestingnamehere]

Okay, I think this is good to go too. Someone with access to the org-level controls will have to go in and enable cirrus ci for this repo though. No idea who can do that or what the process for getting an approval for that looks like though.

Key benefits: Cirrus CI makes it remarkably easy to test on both the Arm/Linux and ARM/OSX configurations. Their Arm/Linux workers seem really backlogged today though, so I'm keeping the builds on that config pretty limited. Upgrading to a paid tier would fix that and their pricing model is pretty reasonable. They also allow requesting workers with more resources, so we could easily use them for that too.

I was able to add a FreeBSD build, but the test failures there are especially dubious. Letting it run as an expected failure or disabling it entirely are both fine options right now. I'm inclined to leave it in since things will likely get better as we fix the various other bugs I've triaged, but I don't feel strongly one way or the other. Currently it's only running sherwood since #162 prevents the timeouts from working with nemesis. This isn't because there are fewer bugs with sherwood though, it's just that #161 consistently prevents the tests from getting far enough to run into the uninterruptible hang state in the CI server. That's... not great, but I still like having info about how software behaves in different environments because it helps with debugging.

I tried adding a MUSL build, but #160 prevented that from working. Offhand, that looks difficult to fix so I'm inclined to leave it alone unless the needed APIs get implemented upstream.

[insertinterestingnamehere]

If the slight delay for Linux/Arm builds ends up being an issue I can pretty easily migrate those builds to CircleCI instead. The really unique thing about Cirrus CI is their support for Apple's M1 chips. The custom resource allocation and FreeBSD support are nice too tho.

[insertinterestingnamehere]

Just confirmed that the builtin timeout on their freebsd images does actually work, it's just not very quick to pick up on a stalled worker. For example, I set the worker to timeout after 5 minutes and it still runs for like 20 anyway.

[insertinterestingnamehere]

The wait time for their arm/linux workers is pretty terrible though. It takes several hours to get something to run. I'm going to check if circleci is any better.

[insertinterestingnamehere]

Yah, the CircleCI workers are dramatically more responsive, but apparently have slightly different failure modes too (#164). I'll just leave them both as-is until we confirm which, if any, of these are okay being used with the main repo. At the moment, CircleCI is way more responsive for arm/linux, but CirrusCI is the only one that supports Apple M1 and FreeBSD.

[insertinterestingnamehere]

At least for now I'm inclined to leave these ARM/Linux builds up since they'll auto-cancel on new pushes anyway. The failure modes for these ARM workers appear different from the ones used by CircleCI and I'd like to understand why before removing the Cirrus builds.

[insertinterestingnamehere]

Okay, this should be working on the main repo once merged.