Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Summarized verification results in VSA, timeless vs. time-sensitive #1207

Open
AdamZWu opened this issue Oct 16, 2024 · 8 comments
Open

Summarized verification results in VSA, timeless vs. time-sensitive #1207

AdamZWu opened this issue Oct 16, 2024 · 8 comments

Comments

@AdamZWu
Copy link
Contributor

AdamZWu commented Oct 16, 2024

VSA is intended to present a summary of verification results, so that consumers (who trust the attestor) can avoid re-perform verification.

However, practically there are two types of verification results:

  • "Timeless" results, such as SLSA build level, source level, etc. Once verified, the conclusions remain valid forever;
  • Time-sensitive results, such as vulnerability analysis assertion, e.g. "Does not contain CVE with severity above 7.0". These results are subject to change at various time horizons (hours, days, etc.)

Is there any guideline / recommendation from SLSA on how to handle the time-sensitive verification results? E.g.:

  • Should VSA be used for storing time-sensitive verification results at all? If so,
  • How should the attestors convey the time sensitivities to the consumers?
@TomHennen
Copy link
Contributor

My recommendation is that the VSA can be used to store these time-sensitive results and that consumers that care about timeliness should consult the VSA's https://slsa.dev/spec/v1.0/verification_summary#timeVerified to determine if they're acceptable. E.g. the VSA results won't expire, but the consumers may have a requirement for newer VSAs.

It's likely desirable for VSA producers to indicate somehow, either in documentation or in the property name, just how recent the data used to assert that property is.

E.g.

VSA issuer: "When we set VULN_SCANNED in verifiedLevels that means there was a vulnerability scan no older than 24 hours from the time encoded in timeVerified."

or

VSA issuer: "When we set VULN_SCANNED_24H in verifiedLevels that means there was a vulnerability scan no older than 24 hours from the time encoded in timeVerified."

Note that SLSA itself doesn't currently have any time-sensitive results, but does allow VSA producers to set fields in VSAs if they wish. So, SLSA itself leaves this undefined and you really just have my advice. :)

@zachariahcox
Copy link
Contributor

I agree with @TomHennen 's take.

I hadn't thought about this scenario much before, but it's pretty important to talk about when a VSA (and maybe which version of the VSA) produces the attestation.
I suspect it will often be the case that the consumer literally cannot reproduce policy or logic evaluated by the VSA issuer.

This could be either because they don't have access to the tooling or they don't know exactly what the VSA does to determine correctness.

@mlieberman85
Copy link
Member

I agree with @TomHennen as well. I think there's a few subtleties around dynamic or temporal data. I'll throw out a few examples:

  • End of life attestation - It's mostly static but might change over time, for example if a group decides to extend support of a project or if the group dissolves.
  • Vulnerability - Similar to what was mentioned by @AdamZWu and @TomHennen. Vulnerabilities are discovered over time and so a vulnerability attestation might be out of date the minute it's created.
  • Attestor effectiveness/capability - What you said @zachariahcox is also important because for example an older SBOM tool might be less accurate or complete than a newer one and even though an SBOM is "static" a newer SBOM generated from the same input data might be more accurate and complete.
  • Where is the software running - In cases of cloud based installations this could vary minute to minute.

It feels like there's a spectrum here of data that mostly stays static to stuff that is very much point in time.


Some food for thought, but slightly tangential: I have found it can be difficult if you have a hierarchy of VSAs. For example:

  • Source VSA
  • Build VSA

Which then get turned into a Final Summary VSA that is a VSA that includes summary data on the source and build. If you have something that is time sensitive some of the time sensitive data in Source and Build might get lost. For example the policies. One of the thing that's not super clear to me is what happens if Source and Build VSA requires the attestations it bases its summary to be let's say no older than 7 days and then the Final Summary VSA requires its data to be no older than 7 days you potentially have a situation where the VSAs are less than 7 days old but the data they refer to are themselves older than 7 days. There's ways to work around this but I think it needs to be part of the conversation on how we should use and treat VSAs.

@trishankatdatadog
Copy link
Member

Interesting discussion!

@mlieberman85 WDYM by "Source and Build VSA" vs "Final Summary VSA"?

My current thinking on the subject is as follows:

  • VSA should contain timestamps
  • Timely attestations should contain timestamps
  • All signatures should furthermore be timestamped so that we have reasonable assurance that an attestation really was produced at that time

Then, it is up to clients to determine how deep down the rabbit hole they want to go. They can check the VSA timestamps alone (along with the timestamped signatures), or also any "downstream" attestations.

@mlieberman85
Copy link
Member

So, in recent discussions we talked about how most likely you would have different VSAs for different parts of the lifecycle to summarize different things. So as source goes from the source to the build if you have lots of attestations related to the source you would want a VSA for all the things related to source. Then you might want all the things related to build when you go to publish the package.

@trishankatdatadog
Copy link
Member

So, in recent discussions we talked about how most likely you would have different VSAs for different parts of the lifecycle to summarize different things. So as source goes from the source to the build if you have lots of attestations related to the source you would want a VSA for all the things related to source. Then you might want all the things related to build when you go to publish the package.

Interesting: so like a VSA per Track, for example? We call this idea partial verification.

@AdamZWu
Copy link
Contributor Author

AdamZWu commented Oct 21, 2024

How about let's establish two separate concepts:

  • The time of attestation (required)
  • The time of expiry (optional)

The time of attestation allows the consumers of attestation to make a subjective decision on whether the attributes being attested fits their ideal time windows (note that they can have more than one time window, such as a tighter window for "best timeliness", a regular window for "acceptable but not ideal", and a wider window of "absolutely must fall within", etc., and make their consumption verification decisions accordingly.)

The time of expiry, however, makes the consumers aware of an objective (to the consumers) advisory of "best before" time advised by the attestor. The consumers mostly likely should handle attestations by taking into account both their own preference above, and the advised deadline here (if present). They could decide to ignore the "best before" advisory -- but some consumers' decision to ignore should not invalidate the need of having this timestamp.

@trishankatdatadog
Copy link
Member

How about let's establish two separate concepts:

  • The time of attestation (required)
  • The time of expiry (optional)

You would need a trusted time of attestation (e.g., with RFC 3161) in order to better trust any time of expiry (otherwise anyone with a compromised key can make it up).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🆕 New
Development

No branches or pull requests

5 participants