Should we capture provenance for the control plane?

[marcelamelara]

Currently the BuildEnv track doesn't require provenance for the control plane because it's not considered part of the build environment per se? But should there be (except for the bootstrapping problem)? One argument in favor is that at BuildEnv L2+, the platform's control plane is no longer trusted.

Originally posted by @arewm in #1244 (comment)

[marcelamelara]

CC @paveliak

[paveliak]

That's an interesting problem. The short answer is "no" and let me elaborate...

Ultimately the objective of Build Environment track is to provide cryptographic evidence that build is running using the specified image. Image itself has provenance and comes from a build pipeline with a SLSA level but since build runs inside the environment (aka "inside the image") we need to rely on extra things to actually attest the environment build runs in.

One option to address that could be to trust control plane to do that and attest the control plane instead. Apple is doing some very interesting work in this area with their Private Compute Cloud. Below I enumerate few challenges that existing build platforms would need to provide answer for and to be honest I think that the bar could be prohibitively high for the majority of them.

In this track we take slightly different approach relying on the hardware root of trust to do the attestation leaving control plane outside of TCB. This allows us to scope attestation to a single machine instead of attesting Control plane (generally a cloud service) that could consist of multiple machines and micro-services. But the downside is we rely on hardware capabilities (mostly SecureBoot and TPM or similar hardware) which may not be present on all platforms.

Another caveat is by treating Control plane as untrusted we have to accept security risks coming from the interaction with the Control plane. For example, we have to trust build request message that initiates the build. Granted, the important properties of this message are included into the build attestation (e.g. commit SHA). And generally we should include into the provenance all input messages coming from the control plan that can influence build results.

I would add another interesting consideration regarding untrusted control plane as we are on this topic. As we boot build environment we cannot expect (untrusted) control plane to attest build environment before executing the job and terminate environment if it failed the attestation. Rather we include hardware-assisted build environment attestation into the build provenance and let build consumer decide on what to do.

Getting back to attesting Control plane... Suppose we request attestation of the Control plane. What kind of challenges come with this request for the build platform providers?

1. There has to be a transparency with the Control plane code. Digests for all builds (and perhaps more than that) should be published in a transparency log. It is not uncommon to make changes to the Cloud services multiple times a day and we can expect constant churn in that log.
2. There has to be a way to maintain request affinity. Cloud services usually run on many nodes (e.g. containers on K8s cluster) and every request may land on a different node. If client attested node 1 and subsequent request lands on node 2 then prior attestation does not make much sense
3. There has to be some guarantees about the inability to access Control plane via a back door (for administrative purpose)
4. Control plane is a stateful service and attesting compute nodes won't be enough, we would need to attest storage nodes too (and technically all the transitive dependencies)
5. Persistent storage introduces the notion of time into the picture. Not only we would need to attest current state of Storage but also give some guarantees that yesterday (and before that) it was in good shape too.

I would be happy to see these questions solved as they may be relevant to other tracks too (Source track? tagging @zachariahcox for it).