Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go faster, Travis CI! #359

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Conversation

jashank
Copy link
Contributor

@jashank jashank commented Feb 27, 2020

Currently, CI cycle times are awful --- and really variable. The longest recent build I've seen is 9h45m21s, and most builds hover between 5h and 9h. Occasionally, the caches will smile on us and bestow a 45-minute-or-less cycle time, and even that's pretty miraculous.

After lots of experimenting, cajoling, and swearing, I think I've got CI cycle times down to about a quarter-hour or less, consistently.

Along the way, I've flensed out a lot of complexity in the build --- for tests, we really don't need to do much more than build the compiler and run the tests --- Cabal shenanigans shouldn't be really necessary, so we can just use Stack here, which is much more amenable to caching.

  • We cache approximately 8 GB, across 12 build combinations. In theory, we could shrink that, but probably not by much (but pulling down and exploding ~700 MB of cached objects is still pretty costly).
  • A cache-hit build takes 3-4 minutes. The majority of that time is spent exploding the cache (~70-100s) and doing builds for ext2 and bilby (~30s each).
  • Cache-miss builds take ~15 minutes, except for Docgent-enabled builds where we spend ~40 minutes recompiling much of Hackage (thanks, Pandoc). Frustratingly, caches are invalidated on PRs and new branches --- but they should inherit caches on their parent branch, so perhaps this won't hurt us too much?
  • builtin-arrays-enabled builds are currently broken, except on Dargent branches --- when those are merged, these builds should be flipped back to required.
  • Cogent ext2fs builds successfully from Cogent down to a kernel module! 🎉
  • BilbyFs doesn't currently compile.

A build that went OK, I guess:

Some builds that did not go well:

For tests, we really don't need to do much more than build the compiler
and run the tests --- Cabal shenanigans shouldn't be really necessary,
so we can just use Stack here, which should handle caching much better.

The `builtin-arrays' builds are currently broken, except on Dargent
branches --- when those are merged, these builds should be flipped back
to required.

 * Cogent ext2fs builds on Travis.
 * BilbyFs doesn't currently compile; it really should...
Copy link

@zilinc zilinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One general concern is that, we claim we support ... versions of GHC (in cogent.cabal) whereas using stack, many of the claimed versions are not tested. Shall we drop the untested versions? Apart from that, I'm happy with all aspects of this change.


extra-package-dbs: []
extra-deps:
- sbv-7.12 # for the `array' branches
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A question: on my branch, I use sbv > 8. Which version of sbv can I have via extra-deps?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only to get the pins right given the current builtin-arrays code. In theory, you can have whatever version you like. I believe sbv-8.x works fine, although I vaguely recall running into issues with crackNum, a sbv dependency, on GHC 8.2; I'd need to test that again to check, and I'd want to make sure stuff still compiles happily in general anyway.

@jashank
Copy link
Contributor Author

jashank commented Feb 28, 2020

@zilinc wrote:

One general concern is that, we claim we support ... versions of GHC (in cogent.cabal) whereas using stack, many of the claimed versions are not tested. Shall we drop the untested versions?

We could test those. However, I believe it's more valuable to instead test the flag combinations (or to instead turn on more flags by default). e.g., how do recursive-types interact with Docgent or the Haskell backend? This would expand our test matrix massively, unfortunately.

I propose we strip back supported versions to the latest on each of the GHC-x.y branches, unless there's good reason to maintain testing against the point-releases.

Given we're also looking at GHC-8.8 support, perhaps this makes a good opportunity to also drop support for 8.2.x?

@jashank jashank force-pushed the jashank/2020/02/27/zoom-zoom-ci branch from bda0cd5 to 9ee1060 Compare March 5, 2020 00:58
 * With +docgent, we use Pandoc, which won't Haddock in Stack.
 * With +haskell-backend, we get the haskell-lexer, which segfaults Haddock.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants