Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Artifact names are always treated as literals, missing files are ignored #183

Open
thp-canonical opened this issue Mar 13, 2024 · 0 comments

Comments

@thp-canonical
Copy link
Contributor

Assuming my spread.yaml contains the following definition:

backends:
    lxd:
        systems:
            - ubuntu-22.04:
                environment:
                  PLATFORM: amd64
            - ubuntu-22.04-arm64:
                environment:
                  PLATFORM: arm64

In a task, I might want to pack a specific artifact that's named after system-specific environment variables:

# ...
execute: |
    snapcraft
    
artifacts:
    - example_1.0_${PLATFORM}.snap
# ...

Also, no wildcard expansion is possible, which could in some cases alleviate the problem of having to resort to environment variables):

# ...

artifacts:
    - example_1.0_*.snap
# ...

The problem seems to be that the list of artifacts is treated as literal string list when files are packed on the test machine:

  • The list of artifacts is read from YAML via type Task struct in spread/project.go (Artifacts [] string)
  • job.Task.Artifacts is accessed in fetchArtifacts() (spread/runner.go)
  • fetchArtifacts() calls client.RecvTar(remoteDir, job.Task.Artifacts, tarw)
  • client.RecvTar() (in spread/client.go) tries hard to treat each artifact name as literal string (including shell-quoting)
  • The tar command executed on the test machine to pack the files will be something along the lines of /bin/tar cJ --sort=name --ignore-failed-read -- 'example_1.0_${PLATFORM}.snap'

As a workaround (because client.RecvTar() uses --ignore-failed-read) one can (try to) list ALL possible values for the artifacts, and spread will ignore invalid artifacts:

# ...

artifacts:
    - example_1.0_amd64.snap
    - example_1.0_arm64.snap
# ...

Because of --ignore-failed-read, Spread won't really complain about any mistyped and unmatched file names, and silently ignore any files it cannot find. Of course, this way of maintaining artifact names is error prone and tedious, as when e.g. a new architecture is added in the future (e.g. riscv64), it's not enough to add a new system to the list of systems for the lxd backend, but every task has to have its artifact list expanded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant