Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

👻 force npm9 during image build #1781

Merged
merged 3 commits into from
Mar 19, 2024
Merged

👻 force npm9 during image build #1781

merged 3 commits into from
Mar 19, 2024

Conversation

sjd78
Copy link
Member

@sjd78 sjd78 commented Mar 16, 2024

Ever since the build container we use [1][2] updated to use npm@10, the image builder github action has encountered a number of problems. The first was solved with #1746. The second more random problem has been with network connections erroring out when installing dependencies.

To work around the network and nofiles errors:

  1. enforce the use of npm@9
  2. pin the Dockerfile base images to specific version tags builder at :1-88, and runtime at :1-93

Additional solutions:

  • Add the force to npm@9 directly in the Dockerfile for use in all circumstances
  • Find a way to use something like verdaccio to proxy/mirror the npmjs repo during the build so npmjs fetches can be pooled across the parallel image builds

[1] registry.access.redhat.com/ubi9/nodejs-18:latest
[2] https://catalog.redhat.com/software/containers/ubi9/nodejs-18/62e8e7ed22d1d3c2dfe2ca01

@sjd78 sjd78 added the cherry-pick/release-0.3 This PR should be cherry-picked to release-0.3 branch. label Mar 16, 2024
@sjd78 sjd78 closed this Mar 16, 2024
@sjd78 sjd78 reopened this Mar 16, 2024
Ever since the build container we use [1] updated to use
`npm@10`, the image builder github action has encountered
a number of problems. The first was solved with konveyor#1746.
The second more random problem has been with network
connections erroring out when installing dependencies.

To work around the network error, force the build to use `npm@9`.

Alternative solutions:
  - Add the force to `npm@9`` directly in the Dockerfile for use
    in all circumstances

  - Use the `:1-88` version of the build container instead of
    `:latest`

  - Find a way to use something like verdaccio to proxy/mirror
    the npmjs repo during the build so npmjs fetches can be
    pooled across the parallel image builds

[1] registry.access.redhat.com/ubi9/nodejs-18:latest

Signed-off-by: Scott J Dickerson <[email protected]>
@sjd78 sjd78 marked this pull request as draft March 19, 2024 16:40
@sjd78 sjd78 marked this pull request as ready for review March 19, 2024 17:50
Copy link
Collaborator

@rszwajko rszwajko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks explicit enough for me:)

@sjd78 sjd78 merged commit da9da4c into konveyor:main Mar 19, 2024
7 checks passed
github-actions bot pushed a commit that referenced this pull request Mar 19, 2024
Ever since the build container we use [1][2] updated to use `npm@10`,
the image builder github action has encountered a number of problems.
The first was solved with #1746. The second more random problem has been
with network connections erroring out when installing dependencies.

To work around the network and nofiles errors:
  1. enforce the use of `npm@9`
  2. pin the Dockerfile base images to specific version tags;
    builder at `:1-88`, and runtime at `:1-93`

Additional solutions:
  - Add the force to `npm@9` directly in the Dockerfile for use in all
    circumstances
  - Find a way to use something like verdaccio to proxy/mirror the npmjs
    repo during the build so npmjs fetches can be pooled across the parallel
    image builds

[1] registry.access.redhat.com/ubi9/nodejs-18:latest
[2] https://catalog.redhat.com/software/containers/ubi9/nodejs-18/62e8e7ed22d1d3c2dfe2ca01

---------

Signed-off-by: Scott J Dickerson <[email protected]>
Signed-off-by: Cherry Picker <[email protected]>
@sjd78 sjd78 added this to the v0.3.1 milestone Mar 19, 2024
sjd78 added a commit that referenced this pull request Mar 19, 2024
Ever since the build container we use [1][2] updated to use `npm@10`,
the image builder github action has encountered a number of problems.
The first was solved with #1746. The second more random problem has been
with network connections erroring out when installing dependencies.

To work around the network and nofiles errors:
  1. enforce the use of `npm@9`
  2. pin the Dockerfile base images to specific version tags;
    builder at `:1-88`, and runtime at `:1-93`

Additional solutions:
  - Add the force to `npm@9` directly in the Dockerfile for use in all
    circumstances
  - Find a way to use something like verdaccio to proxy/mirror the npmjs
    repo during the build so npmjs fetches can be pooled across the parallel
    image builds

[1] registry.access.redhat.com/ubi9/nodejs-18:latest
[2] https://catalog.redhat.com/software/containers/ubi9/nodejs-18/62e8e7ed22d1d3c2dfe2ca01

---------

Signed-off-by: Scott J Dickerson <[email protected]>
Signed-off-by: Cherry Picker <[email protected]>

Signed-off-by: Scott J Dickerson <[email protected]>
Signed-off-by: Cherry Picker <[email protected]>
Co-authored-by: Scott Dickerson <[email protected]>
@ibolton336
Copy link
Member

ibolton336 commented Mar 25, 2024

Looks like this is causing NPM install to fail locally. Error fixed by moving back to
"npm": "^9.5.0" in package.json

Error:

➜  tackle2-ui git:(main) npm i
npm ERR! code EBADENGINE
npm ERR! engine Unsupported engine
npm ERR! engine Not compatible with your version of node/npm: @konveyor-ui/[email protected]
npm ERR! notsup Not compatible with your version of node/npm: @konveyor-ui/[email protected]
npm ERR! notsup Required: {"node":">=18.14.2","npm":"^9.5.0"}
npm ERR! notsup Actual:   {"npm":"10.2.0","node":"v21.1.0"}

npm ERR! A complete log of this run can be found in: /Users/ibolton/.npm/_logs/2024-03-25T13_35_37_824Z-debug-0.log
➜  tackle2-ui git:(main)

@ibolton336
Copy link
Member

This was due to my local npm version falling outside the supported version range in package.json. Fixed with:
npm install -g [email protected]

@sjd78 sjd78 deleted the force_npm9 branch April 19, 2024 17:09
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request Apr 22, 2024
Backport image build changes made on main and release-0.3
to get release-0.2 building properly.

See: konveyor#1781, konveyor#1780
Follow-up to konveyor#1801, konveyor#1871

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request Apr 22, 2024
Backport image build changes made on main and release-0.3
to get release-0.2 building properly.

See: konveyor#1781, konveyor#1780
Follow-up to konveyor#1801, konveyor#1871

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit that referenced this pull request Apr 30, 2024
Backport image build changes made on main and release-0.3 to get
release-0.2 building properly.

See: #1781, #1780
Follow-up to #1801, #1871

---------

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile`, then run image builds
for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any `Dockerfile` change
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

In future, it may be reasonable to extend this check
to happen when other core build related changes are
made (package-lock.json, package.json).

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 10, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 13, 2024
If the workflow is run from a PR, and the PR includes
a change to the `Dockerfile` or `package-lock.json`, then
run image builds for all of our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes
will not cause the image-build-and-push on PR merge workflow
to break.  Doing the image build here should show up most
problems much earlier.  For example, a npm version update
in the build container could break github action `nofiles`
or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of
when this check could have caught issues before a PR
merge.

Supports: konveyor#1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit that referenced this pull request May 14, 2024
If the workflow is run from a PR, and the PR includes a change to the
`Dockerfile` or `package-lock.json`, then run image builds for all of
our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes will not
cause the image-build-and-push on PR merge workflow to break. Doing the
image build here should reveal most problems much earlier. For example,
a npm version update in the build container could break github action
`nofiles` or network access capabilities for the npm install.

See #1742, #1746, and #1781 for some other examples of when this check
could have caught issues before a PR merge.

Supports: #1883

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 15, 2024
…ange (konveyor#1907)

If the workflow is run from a PR, and the PR includes a change to the
`Dockerfile` or `package-lock.json`, then run image builds for all of
our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes will not
cause the image-build-and-push on PR merge workflow to break. Doing the
image build here should reveal most problems much earlier. For example,
a npm version update in the build container could break github action
`nofiles` or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of when this check
could have caught issues before a PR merge.

Supports: konveyor#1883
Backport-of: konveyor#1907

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit to sjd78/tackle2-ui that referenced this pull request May 15, 2024
…ange (konveyor#1907)

If the workflow is run from a PR, and the PR includes a change to the
`Dockerfile` or `package-lock.json`, then run image builds for all of
our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes will not
cause the image-build-and-push on PR merge workflow to break. Doing the
image build here should reveal most problems much earlier. For example,
a npm version update in the build container could break github action
`nofiles` or network access capabilities for the npm install.

See konveyor#1742, konveyor#1746, and konveyor#1781 for some other examples of when this check
could have caught issues before a PR merge.

Supports: konveyor#1883
Backport-of: konveyor#1907

Note: build architectures updated to match the settings on the
      backport target branch

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit that referenced this pull request May 15, 2024
…ange (#1907) (#1914)

If the workflow is run from a PR, and the PR includes a change to the
`Dockerfile` or `package-lock.json`, then run image builds for all of
our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes will not
cause the image-build-and-push on PR merge workflow to break. Doing the
image build here should reveal most problems much earlier. For example,
a npm version update in the build container could break github action
`nofiles` or network access capabilities for the npm install.

See #1742, #1746, and #1781 for some other examples of when this check
could have caught issues before a PR merge.

Supports: #1883
Backport-of: #1907

Signed-off-by: Scott J Dickerson <[email protected]>
sjd78 added a commit that referenced this pull request May 15, 2024
…ange (#1907) (#1915)

If the workflow is run from a PR, and the PR includes a change to the
`Dockerfile` or `package-lock.json`, then run image builds for all of
our target platforms.

The images are built but not pushed to any repository.

We want to be reasonably sure that any major build file changes will not
cause the image-build-and-push on PR merge workflow to break. Doing the
image build here should reveal most problems much earlier. For example,
a npm version update in the build container could break github action
`nofiles` or network access capabilities for the npm install.

See #1742, #1746, and #1781 for some other examples of when this check
could have caught issues before a PR merge.

Supports: #1883
Backport-of: #1907

Note: build architectures updated to match the settings on the backport
target branch

Signed-off-by: Scott J Dickerson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick/release-0.3 This PR should be cherry-picked to release-0.3 branch.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants