content: source track: update source threats for draft spec

[zachariahcox]

fixes: #1178

The threats.md file is intended to be a one-stop summary of all the threats mitigated by SLSA.
From this page, we should plan to redirect to other parts of the documentation.

[netlify]

✅ Deploy Preview for slsa ready!

Name	Link
🔨 Latest commit	97094b5
🔍 Latest deploy log	https://app.netlify.com/sites/slsa/deploys/6750d74e3cc0b50008496c87
😎 Deploy Preview	https://deploy-preview-1236--slsa.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

Copilot reviewed 1 out of 1 changed files in this pull request and generated no suggestions.

Comments skipped due to low confidence (2)

docs/spec/draft/threats.md:86

• [nitpick] The word 'SHOULD' should not be in all caps unless it is a specific requirement in a specification. Consider changing it to 'should'.

Trustworthiness scales with transparency, and consumers SHOULD push on their vendors to follow transparency best-practices.

docs/spec/draft/threats.md:86

• [nitpick] The phrase 'Trustworthiness scales with transparency' is unclear. Consider rephrasing it to 'Trustworthiness increases with transparency'.

Trustworthiness scales with transparency, and consumers SHOULD push on their vendors to follow transparency best-practices.

Tip: Turn on automatic Copilot reviews for this repository to get quick feedback on every pull request. Learn more

[TomHennen]

Just the one comment, otherwise this looks great!

[adityasaky]

this isn't a complete review because I wasn't sure if more changes are coming, so I'm submitting this for visibility for now.

[adityasaky]

Suggested change

	Tools like the [OSSF Scorecard](https://github.com/ossf/scorecard) can help to quantify the risk of consuming artifacts from specific organizations, but do not fully remove it.
	Tools like the [OSSF Scorecard](https://github.com/ossf/scorecard) can help to quantify the risk of consuming artifacts from specific producers, but do not fully remove it.

Also, can we point to some specific ways to use scorecard here? I think we want to be careful with what we say can help with something like the xz attack...

[adityasaky]

I wonder if this can lean on the source track and point to the documented change management process piece for setting guardrails on submitting changes.

[zachariahcox]

I think in many ways this section is duplicative of other sections.

We could just say:

*Threat*: source threats!
*Mitigation*: SLSA source track! <url>

[zachariahcox]

@adityasaky do you think we should just cut all these scenarios and redirect to the source track?

[TomHennen]

Note that this page discusses lots of threats to the build and doesn't just defer to the SLSA Build Track.

The general purpose of this page, AIUI, was to centralize all the various threats in one place and direct folks elsewhere as needed.

[zachariahcox]

Potentially then most of these examples should link to relevant sections of the relevant tracks. Maybe we'll take it as a todo.

[trishankatdatadog]

"Transparency" is unfortunately an overloaded word in our world. WDYM specifically in this context?

[zachariahcox]

oof, that's a good callout.

I propose that I mean:

• open source
• open build
• open policies
• published "definitions of correctness."

[TomHennen]

I'd be inclined to remove the sentence all together. Yes, transparency (visibility into the process that creates the software) is one way of increasing trustworthiness but there are other vectors too. "Does this organization have something to lose if they do a bad thing? Do I think that's enough of an incentive for them not to do the bad thing".

Additionally, I don't think "Transparency best-practices" are well defined anywhere (or even vaguely agreed upon...).

Perhaps the easiest way forward is

1. to focus on the submission of bad source (so open builds don't apply here) and assume the other parts of the chain are protected (at the very least they're covered by other threats on this page).
2. to be a bit more hand-wavy since this is a squishy subject:

**Mitigation**: Users must establish some basis to trust the organization they are consuming software from.  That basis may be 1) that the code is open sourced **and** has a sufficiently large user-base that makes it more likely for malicious changes to be detected, 2) that the organization has sufficient incentives (legal, reputational, etc.) to dissuade it from making malicious changes.  Ultimately this is a judgement call with no straightforward answer.

[zachariahcox]

@TomHennen I think SLSA tooling gives you a way to unilaterally improve trust when the source and builds are open source. This is possible independently of whether or not the producer has something to lose.

I agree though that this section needs to be reduced.

[zachariahcox]

I left two suggestions. wdyt @trishankatdatadog @TomHennen ?

[trishankatdatadog]

Left a comment down there...

[TomHennen]

I'd be inclined to remove the sentence all together. Yes, transparency (visibility into the process that creates the software) is one way of increasing trustworthiness but there are other vectors too. "Does this organization have something to lose if they do a bad thing? Do I think that's enough of an incentive for them not to do the bad thing".

Additionally, I don't think "Transparency best-practices" are well defined anywhere (or even vaguely agreed upon...).

Perhaps the easiest way forward is

1. to focus on the submission of bad source (so open builds don't apply here) and assume the other parts of the chain are protected (at the very least they're covered by other threats on this page).
2. to be a bit more hand-wavy since this is a squishy subject:

**Mitigation**: Users must establish some basis to trust the organization they are consuming software from.  That basis may be 1) that the code is open sourced **and** has a sufficiently large user-base that makes it more likely for malicious changes to be detected, 2) that the organization has sufficient incentives (legal, reputational, etc.) to dissuade it from making malicious changes.  Ultimately this is a judgement call with no straightforward answer.

[TomHennen]

If we merge this before 1.1 goes out will that make it harder to remove this from the 1.1 release? Have we decided we want to include this in the 1.1 release (fine with me at this point)?

@lehors thoughts?

[zachariahcox]

@TomHennen what if we update this to cover both:

Suggested change

	*Threat:* A producer intentionally creates a malicious revision with the intent of harming their consumers.
	*Threat:* A producer intentionally creates a malicious revision (or a VSA issuer intentionally creates a malicious attestation) with the intent of harming their consumers.

The mitigation for both is a clearer -- you cannot.

[zachariahcox]

Suggested change

	Trustworthiness scales with transparency, and consumers SHOULD push their vendors to follow transparency best-practices.
	When transparency is not possible, consumers may choose not to consume the artifact, or may require additional evidence of correctness from a trusted third-party.
	Tools like the [OSSF Scorecard](https://github.com/ossf/scorecard) can help to quantify the risk of consuming artifacts from specific producers, but do not fully remove it.
	For example, a consumer may choose to only consume source artifacts from repositories that have a high score on the OSSF Scorecard.
	Ultimately, consumers must decide which producers are trusted to produce artifacts and which issuers are trusted to produce VSAs.

We could redirect from here over to "verifying platforms".

[trishankatdatadog]

Sorry for late reply. Why do consumers even need to know which producers are trusted to produce artifacts (source code, I imagine, in this case)? Couldn't they just get away with the Source Track VSA from the issuer, and call it a day so long as its subjects/outputs (e.g., source code) matches the inputs to the next step (e.g., build)?

[zachariahcox]

this wording aligns better with the build integrity section below.

[zachariahcox]

we could add another section here like

*threat*: A member of the producer's organization acts against the intent of the producer.

We could say this is when "the producer UNintentially ships bad code."

[zachariahcox]

Suggested change

	Intermediate revisions don't count as being reviewed and are not added to the protected context (such as the `main` branch).
	In this configuration, intermediate revisions are not considered to be approved and are not added to the protected context (e.g. the `main` branch).
	With git version control systems this is called a "squash" merge strategy.

[zachariahcox]

I do not think we should use "two trusted persons" as a proxy for producer intent. We should probably say "the documented change process for the source" if we must, but I prefer just "the intent".

[zachariahcox]

If we merge this before 1.1 goes out will that make it harder to remove this from the 1.1 release? Have we decided we want to include this in the 1.1 release (fine with me at this point)?

@lehors thoughts?

@TomHennen! Potentially I am off base -- is 1.1 going to be different from the /draft folder?
I think this kind of change should go out with the most recent version of the source track, whichever that is 👍
It was not my intention to add items to the 1.1 release if that is different than the drafts folder.
In the meantime I'll update the title to talk about drafts spec instead of 1.1

[lehors]

If we merge this before 1.1 goes out will that make it harder to remove this from the 1.1 release? Have we decided we want to include this in the 1.1 release (fine with me at this point)?
@lehors thoughts?

@TomHennen! Potentially I am off base -- is 1.1 going to be different from the /draft folder? I think this kind of change should go out with the most recent version of the source track, whichever that is 👍 It was not my intention to add items to the 1.1 release if that is different than the drafts folder. In the meantime I'll update the title to talk about drafts spec instead of 1.1

I'm glad @TomHennen remembers that we need to get 1.1 finalized. I was going to bring that up myself looking at this PR. Unfortunately, we don't have a very clean way of working on several versions in parallel.

The plan was to only work in the draft folder and when all 1.1 issues have been addressed to create a new folder for 1.1 selecting (manually) from the draft what is relevant. Of course this is really only practical if the differences are fairly coarse (like whole files to be left out). If the content of some files start changing in ways that are only relevant for the next version this operation becomes quite impractical.

If we want to go ahead and merge this, I will need to create the 1.1 folder before, and from then on every change made for 1.1 will have to be made in both folders (1.1 and draft). Not ideal but doable.

[TomHennen]

If we merge this before 1.1 goes out will that make it harder to remove this from the 1.1 release? Have we decided we want to include this in the 1.1 release (fine with me at this point)?
@lehors thoughts?

@TomHennen! Potentially I am off base -- is 1.1 going to be different from the /draft folder? I think this kind of change should go out with the most recent version of the source track, whichever that is 👍 It was not my intention to add items to the 1.1 release if that is different than the drafts folder. In the meantime I'll update the title to talk about drafts spec instead of 1.1

I'm glad @TomHennen remembers that we need to get 1.1 finalized. I was going to bring that up myself looking at this PR. Unfortunately, we don't have a very clean way of working on several versions in parallel.

The plan was to only work in the draft folder and when all 1.1 issues have been addressed to create a new folder for 1.1 selecting (manually) from the draft what is relevant. Of course this is really only practical if the differences are fairly coarse (like whole files to be left out). If the content of some files start changing in ways that are only relevant for the next version this operation becomes quite impractical.

If we want to go ahead and merge this, I will need to create the 1.1 folder before, and from then on every change made for 1.1 will have to be made in both folders (1.1 and draft). Not ideal but doable.

I wonder if at this point we can just wait and include the source track (and the build track if it's ready). We've waited this long, and there don't seem to be that many outstanding issues elsewhere. By trying to separate them now we'd probably just cause ourselves more work for less gain. WDYT?

[lehors]

I wonder if at this point we can just wait and include the source track (and the build track if it's ready). We've waited this long, and there don't seem to be that many outstanding issues elsewhere. By trying to separate them now we'd probably just cause ourselves more work for less gain. WDYT?

You mean to skip 1.1? That would solve the problem of course. It's true that the more time goes by the less relevant it is.

[TomHennen]

I wonder if at this point we can just wait and include the source track (and the build track if it's ready). We've waited this long, and there don't seem to be that many outstanding issues elsewhere. By trying to separate them now we'd probably just cause ourselves more work for less gain. WDYT?

You mean to skip 1.1? That would solve the problem of course. It's true that the more time goes by the less relevant it is.

Not so much "skip" as include the source track in 1.1. New tracks are supposed to go in minor releases

[TomHennen]

We had a long, long, chat about tracks, threats pages, etc... at yesterday's meeting. Do we still like this PR or do we want to simplify it some? I have no opinion at this time.