Add CI caching and improve CI tooling

[Eduardogbg]

• fix our rust toolchain so it's not divergent between local and ci anymore; previously the local toolchain would be 1.90 but ci would use nightly features to format the code. to preserve the formatting options i opted into picking a nightly version close to 1.90 for formatting on ci and also overall local version, and 1.90 for other ci jobs.
• install lake dependencies so we can use lake exe cache get
• use lake exe cache get to avoid rebuilding mathlib, lampe and stdlib
• fix out of disk issues on ci
• create shared container image for all jobs, with dev dependencies already installed
  • xonsh and python deps
  • lake and c++ deps
  • cargo

[iamrecursion]

We're very explicitly building with a stable release, so it should be used for local development and for CI builds. The only reason we use a nightly for formatting is that many formatting config options are unstable only. You can run those locally for formatting with cargo +nightly fmt if you have rustup installed.

[iamrecursion]

This is a really great start! Thanks so much for cleaning this up bigtime! A few comments:

• What is the impact to our CI runtime to use the container? I would hope that even with that new overhead it reduces the general CI runtime.
• Please make sure that there are as few mentions of the lean toolchain version or rust toolchain version as possible. If you can, I would suggest relying only on the rust-toolchain.toml and lean-toolchain files as relevant.
• If that is not possible, we should be checking that they are coherent in any place that they exist on CI and before building any images.

I do apologise if I've left any silly questions. CI and particularly containers are quite far out of my wheelhouse, so please point out anywhere I've got something wrong!

Other than that, please let me know if you've got any questions, and thanks for the hard work at all of this!

[iamrecursion]

Just a thought: an interesting test for whether this all works with good devx would be trying to update the Rust version to the latest stable release and separately the Lean version to the latest stable release.

Do note that for the Lean version #234 is worth taking a look at as that claims to have the fix we need for the issue that was blocking us from versions >=4.22.x.

[iamrecursion]

When you want me to take a look at this again, please ping me and/or re-request review! Easiest way to get my attention!

I've also fixed the branch protection rules to look for the "Build CLI" job name instead of "Build Rust" so it shouldn't be blocked on the CI portion any more, just an approving review.

[Eduardogbg]

a lot of comments I still need to address but at least now we have the image already pushed to reilabs/lampe-ci:rust-1.90.0-lean-v4.21.0-noir-9a5b

shame i didn't realize earlier i could have just built and pushed the image from this PR, I'll close #240 now, have already rebased this branch on top of those changes.

about your worries if this will still be faster after the image build step: yes, because we won't rebuild the image on every PR. and other changes about making the DAG better and hitting cache whenever possible go a long mile.

we have to decide on when we want to build the image but i think we can just run manually whenever we want to change the versions. requiring to merge something first to only then be able to update the image version could be annoying. but we can always check if the toolchain versions are all the same and if not, build and push a new image.

[iamrecursion]

This is really good stuff. A few more questions and comments about making this work smoothly!

[Eduardogbg]

I went back and forth with how to structure the commits, removed the previous discarded attempts from history, but still not suuuper happy with how it turned out, wish the scripts could be simplified and didn't depend on different logic for CI, but there are a few footguns this avoids (i think lake will verify mathlib on ci or something like that, at least i ran into that when iterating earlier)

i sugggest we squash the commits before merging. maybe something like

ci: use image with build dependencies, hit cache when possible and structure dag

- creates new docker image with all of lampe's build & test dependencies to run on ci
- have all ci jobs depend on the new image
- deduplicate all lean and rust builds by structuring jobs dag correctly and restoring cache

[iamrecursion]

This is fantastic! Thanks so much for all the hard work on this!

[iamrecursion]

I take it we don't need the GHCR_TOKEN now?

[Eduardogbg]

precisely. we needed it temporarily cuz i wasnt sure i had permissions to push images from actions in my pr before it got merged. but in hindsight it was never necessary

[iamrecursion]

Awesome. I'll remove it from the repo!