[feature] Generator build level

[laurentsimon]

The new v1.0 specs, iiuc, no longer has a "provenance" level 3. I think this means the generators would become level 2.
We could probably make them level 3 if the generator is able to check tat the caller workflow uses only GitHub-hosted runners.

Is there a way to do that?

The list of GitHub runners is available at https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources.

Is there an API to retrieve that list at runtime? If so, we could compare it to the list of runners used in the workflow.

There's also this API https://stackoverflow.com/questions/70027430/get-a-list-of-github-runners, which I have not tried. It may require org-level tokens to query

[laurentsimon]

Follow-up. If we're able to do that, the generator may need to have different buildTypes, one for level 3 and one for level 2 (self-hosted). Or we even call them generator/self-hosted@vx and generator/github-hosted@vx

[asraa]

Might be possible to detect self-hosted label here: https://docs.github.com/en/actions/hosting-your-own-runners/using-self-hosted-runners-in-a-workflow#using-default-labels-to-route-jobs

[laurentsimon]

Great finding, thanks.

The API https://docs.github.com/en/rest/actions/workflow-jobs?apiVersion=2022-11-28#get-a-job-for-a-workflow-run--code-samples seems to return the labels, so I think we should be able to list the workflow run's jobs and use the label to identify self vs GH-hosted runners

[laurentsimon]

we can always fetch the workflows and parse them ourselves if we want

[laurentsimon]

@ramonpetgrave64 that's an important issue we have not had the time to resolve for a while

[ramonpetgrave64]

It should also be indicated in the OIDC token at runner_environment that we might fetch

• https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#understanding-the-oidc-token
• https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#updating-your-actions-for-oidc

[laurentsimon]

the OIDC token we fetch would happen in the context of the generator, not the calling repository, no?

Additional notes for implementation. I think we need to be able to ensure that all (reusable) workflows referenced by the user workflow contain the same runner, not only the workflow that are part of the "call stack".

[ramonpetgrave64]

It seems like the labels alone are not a good way to detect self-hosted runners. I've done some experimenting and found that you can still target your Jobs for self-hosted runners without explicitly using the label "self-hosted": just label your runners something else.

Here I have a self-hosted runner I named "Github Actions" and with a label "ubuntu-latest".

I'm going to look more into this: List Jobs for Workflow also returns the runners' IDs, so maybe I could use that with List Self-hosted Runners

@laurentsimon I'll have to look more into the OIDC token, too. But I think you're probably right that it would be in the context of the re-usable workflow as it's being executed on a self-hosted runner.

[laurentsimon]

It seems like the labels alone are not a good way to detect self-hosted runners. I've done some experimenting and found that you can still target your Jobs for self-hosted runners without explicitly using the label "self-hosted": just label your runners something else.

Is there a way too access the self-hosted info from your label?

I'm going to look more into this: List Jobs for Workflow also returns the runners' IDs, so maybe I could use that with List Self-hosted Runners

You're trying to check if a runner if self-hosted. How about checking whether the runner is GitHub-hosted, and treat everything else a self-hosted? Would that be easier?

[laurentsimon]

the API contains a "runner_name": "my runner",. What does this contain? Could we use this API or a similar one to check if the name is not on of the GitHub-hosted runners?

[ramonpetgrave64]

We can't use runner_name. All the github-hosted runners will be named something like Github Action 2 or Github Actions 15, but I could also give my self-hosted runner a similar name. There may be something to take advantage of with runner_group_name, but that's a paid feature I can't test yet.

Example of a github-hosted runner's response:

{
...
      runner_id: 54,
      runner_name: 'GitHub Actions 23',
      runner_group_id: 2,
      runner_group_name: 'GitHub Actions'
    },

[laurentsimon]

Thanks for the info. In the API doc, I see a response example containing:

"labels": [
    "self-hosted",
    "foo",
    "bar"
  ],

Is this not what we need? is it present in the response you get?

[ramonpetgrave64]

Yes, but those are the labels that the calling workflow job specifies, not necessarily all the labels that a runner possesses.
So, in my workflow yaml, I can have a job that says runs-on: ubuntu-latest, then the ListJobs Api would only return "labels": ["ubuntu-latest"] for that job, meanwhile the job executes on my self-hosted runner that has my custom label ubuntu-latest.

[laurentsimon]

I thought it's a requirement for a self-hosted runner to have a label self-hosted, otherwise it's never routed to a self-hosted runner? Is that assumption not correct?

[laurentsimon]

See https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/using-self-hosted-runners-in-a-workflow#about-self-hosted-runner-labels

All self-hosted runners have the self-hosted label. Using only this label will select any self-hosted runner. To select runners that meet certain criteria, such as operating system or architecture, we recommend providing an array of labels that begins with self-hosted (this must be listed first) and then includes additional labels as needed. When you specify an array of labels, jobs will be queued on runners that have all the labels that you specify.

[ramonpetgrave64]

@laurentsimon I think I explained it a bit better in my PR description #3298

[ramonpetgrave64]

Their next paragraph says that the user doesn't necessarily have to specify the label "self-hosted" in order to use the self-hosted runners, neither in the workflow spec nor as a property of the runners themselves.

Although the self-hosted label is not required, we strongly recommend specifying it when using self-hosted runners to ensure that your job does not unintentionally specify any current or future GitHub-hosted runners.

[laurentsimon]

I see. It's really confusing: Although the self-hosted label is not required and All self-hosted runners have the self-hosted label contradict each other. I think you verified that self-hosted is not necessary? Would be good to file an issue and get confirmation from them to remove the confusion in the doc

[ramonpetgrave64]

Yes I verified that the user doesn't have to specify runs-on: "self-hosted" in order to use their self-hosted runners. They can do runs-on: "cloudtop" or even runs-on: "ubuntu-latest", if they configure their runners to have those custom labels.

[ramonpetgrave64]

I've been advised to open a ticket with Gtihub, asking for their ListJobsForWorklfowRun to have fore definitive information to solve our problem.
https://github.com/orgs/community/discussions/111347