ci: Merge qemu & metal tests

[travier]

Test PR to see how long it takes to run both qemu & metal tests in a single job (would save us a build of RHCOS).
/hold

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: travier

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [travier]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@travier: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/rhcos-86-build-test-qemu	440bb93	link	true	/test rhcos-86-build-test-qemu

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[travier]

@jlebon: This one failed as it lasted more than 3h

[travier]

We could increase the timeout, but I don't think it's a great experience when tests last more than 3h.

[cgwalters]

Yeah to really make this sustainable we need to figure out how to get Prow jobs to actually share the ostree-container build.

Worst case I think we could do something hacky like push the image to a (separate) registry, and then have the other jobs poll/wait for it.

Once we have that, we can "shard" tests as much as we want. For example,

• add a --shard=N option to kola
• Split up the kola tests into 2 shards i.e. rhcos-86-test-1 and rhcos-86-test-2 prow contexts

Also, I still think we should get away from running all tests on all builds. We also shouldn't only run the tests on qemu, now that we're in Prow we've always been able to use the Prow cloud credentials to e.g. run tests on AWS/GCP/etc. Splitting off that alone, particularly to use the new kola run --oscontainer flow coreos/coreos-assembler#3004 would be an entirely different way to shard and speed up tests.

[openshift-merge-robot]

@travier: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci]

@openshift-bot: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.