Changed to execute a scan job when the build job is finished. #340

012xx · 2024-11-15T06:41:02Z

Overview

Changed to execute a scan job when the build job is finished.

Why

Due to the increasing frequency of yamory scan failures

What

I did not want to execute curl -sSf -L https://mw-receiver.yamory.io/image/script/trivyin in parallel.
For this reason, I used a for loop and other methods to execute the scan sequentially.
I also made changes to use the needs context so that the scan is executed after the build is complete.
Due to the separation of jobs, it became difficult to pass data in the BUILDS file created during the build, so I decided to execute the scan by putting the image name in an environment variable.

takahiro-yamada · 2024-11-15T08:15:43Z

.github/workflows/release.yaml

+  scan:
+    name: Scan images
+    runs-on: ubuntu-24.04
+    needs: build
+    steps:


By splitting a job, we can't refer BUILD file generated at build job at scan job.
You can use GITHUB_OUTPUT variable to pass key-value pairs across jobs.

.github/workflows/release.yaml

takahiro-yamada

As mentioned at other place, we need to consider how to use GITHUB_OUTPUT

…ase workflow

.github/workflows/release.yaml

takahiro-yamada · 2024-11-19T05:10:28Z

.github/workflows/release.yaml

-              YAMORY_IMAGE_IDENTIFIER="${d}:$BRANCH" YAMORY_IMAGE_NAME="${d}:$TAG" bash -c "$(curl -sSf -L https://mw-receiver.yamory.io/image/script/trivy)"
-          done
+            for ubuntu_version in ${{ env.ubuntu-version }}; do
+              cd $ubuntu_version


[Q]
Does this work?
It may fail from second loop because the second time it run the cd command from ${WORK_DIR}/20.04 to ${WORK_DIR}/20.04/22.04.

I found out that it didn't work when I debugged it on my own, so I fixed the way I wrote it 🙏

…rkflow

takahiro-yamada · 2024-11-20T02:15:36Z

.github/workflows/release.yaml

@@ -6,8 +6,8 @@ on:
 env:
  go-version: 1.23
 jobs:
-  release:
-    name: release images
+  build:


It looks better the job identifier remain "release" since its main purpose is release.

@012xx
How about this?

takahiro-yamada · 2024-11-20T02:18:41Z

.github/workflows/release.yaml

+            cd ${ubuntu_version}
+            TAG=$(cat TAG)
+            for ubuntu_image in ${{ env.ubuntu-image }}; do
              echo
-              echo "scanning $d:$TAG ..."
-              YAMORY_IMAGE_IDENTIFIER="${d}:$BRANCH" YAMORY_IMAGE_NAME="${d}:$TAG" bash -c "$(curl -sSf -L https://mw-receiver.yamory.io/image/script/trivy)"
+              echo "scanning ${ubuntu_image}:${TAG} ..."
+              YAMORY_IMAGE_IDENTIFIER="ghcr.io/cybozu/${ubuntu_image}:${ubuntu_version}" YAMORY_IMAGE_NAME="${ubuntu_image}:${TAG}" bash -c "$(curl -sSf -L https://mw-receiver.yamory.io/image/script/trivy)"
+            done
+            cd ../


As mentioned another PR, we don't need to cd but only specify the paths from parent path.

e.g., TAG=$(cat "${ubuntu_version}/TAG")

…n in release workflow

takahiro-yamada · 2024-11-20T02:49:06Z

.github/workflows/release.yaml

-              echo "scanning $d:$TAG ..."
-              YAMORY_IMAGE_IDENTIFIER="${d}:$BRANCH" YAMORY_IMAGE_NAME="${d}:$TAG" bash -c "$(curl -sSf -L https://mw-receiver.yamory.io/image/script/trivy)"
+              echo "scanning ${ubuntu_image}:${TAG} ..."
+              YAMORY_IMAGE_IDENTIFIER="ghcr.io/cybozu/${ubuntu_image}:${ubuntu_version}" YAMORY_IMAGE_NAME="${ubuntu_image}:${TAG}" bash -c "$(curl -sSf -L https://mw-receiver.yamory.io/image/script/trivy)"


I think YAMORY_IMAGE_NAME should be modified.
Please check the file before you modified here.

Suggested change

YAMORY_IMAGE_IDENTIFIER="ghcr.io/cybozu/${ubuntu_image}:${ubuntu_version}" YAMORY_IMAGE_NAME="${ubuntu_image}:${TAG}" bash -c "$(curl -sSf -L https://mw-receiver.yamory.io/image/script/trivy)"

YAMORY_IMAGE_IDENTIFIER="ghcr.io/cybozu/${ubuntu_image}:${ubuntu_version}" YAMORY_IMAGE_NAME="ghcr.io/cybozu/${ubuntu_image}:${TAG}" bash -c "$(curl -sSf -L https://mw-receiver.yamory.io/image/script/trivy)"

takahiro-yamada · 2024-11-20T04:10:18Z

.github/workflows/release.yaml

@@ -6,8 +6,8 @@ on:
 env:
  go-version: 1.23
 jobs:
-  release:
-    name: release images
+  build:


@012xx
How about this?

takahiro-yamada

LGTM

fix: Improving the scan job

7afa25a

012xx requested a review from takahiro-yamada November 15, 2024 06:41

takahiro-yamada requested changes Nov 15, 2024

View reviewed changes

.github/workflows/release.yaml Show resolved Hide resolved

012xx added 2 commits November 15, 2024 08:47

Add: GITHUB_OUTPUT

3b1ea8c

Add: working-directory

a658672

012xx requested a review from takahiro-yamada November 15, 2024 08:54

takahiro-yamada requested changes Nov 19, 2024

View reviewed changes

012xx added 2 commits November 19, 2024 02:19

refactor: Simplify image scanning logic and environment setup in rele…

d6599dc

…ase workflow

remove: GITHUB_OUTPUT

2a2fd83

012xx requested a review from takahiro-yamada November 19, 2024 02:25

012xx changed the title ~~Change to scan job when the build job is finished.~~ Changed to execute a scan job when the build job is finished. Nov 19, 2024

takahiro-yamada requested changes Nov 19, 2024

View reviewed changes

012xx added 2 commits November 19, 2024 05:27

fix: indent

f27f2f9

fix: Add checkout step and correct directory navigation in release wo…

8b47ac9

…rkflow

012xx requested a review from takahiro-yamada November 19, 2024 11:18

takahiro-yamada requested changes Nov 20, 2024

View reviewed changes

fix: Correct TAG file path and remove unnecessary directory navigatio…

134213f

…n in release workflow

012xx requested a review from takahiro-yamada November 20, 2024 02:46

takahiro-yamada requested changes Nov 20, 2024

View reviewed changes

fix: YAMORY_IMAGE_NAME

62b1e5c

012xx self-assigned this Nov 20, 2024

012xx requested a review from takahiro-yamada November 20, 2024 03:15

takahiro-yamada requested changes Nov 20, 2024

View reviewed changes

fix: jobname

ec9626d

012xx requested a review from takahiro-yamada November 20, 2024 04:37

takahiro-yamada approved these changes Nov 20, 2024

View reviewed changes

012xx merged commit 70dbf40 into main Nov 20, 2024
3 checks passed

012xx deleted the scan-compatible branch November 20, 2024 05:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed to execute a scan job when the build job is finished. #340

Changed to execute a scan job when the build job is finished. #340

012xx commented Nov 15, 2024 •

edited

Loading

takahiro-yamada Nov 15, 2024

takahiro-yamada left a comment

takahiro-yamada Nov 19, 2024

012xx Nov 19, 2024

takahiro-yamada Nov 20, 2024

takahiro-yamada Nov 20, 2024

takahiro-yamada Nov 20, 2024

takahiro-yamada Nov 20, 2024

takahiro-yamada Nov 20, 2024

takahiro-yamada left a comment

	YAMORY_IMAGE_IDENTIFIER="ghcr.io/cybozu/${ubuntu_image}:${ubuntu_version}" YAMORY_IMAGE_NAME="${ubuntu_image}:${TAG}" bash -c "$(curl -sSf -L https://mw-receiver.yamory.io/image/script/trivy)"
	YAMORY_IMAGE_IDENTIFIER="ghcr.io/cybozu/${ubuntu_image}:${ubuntu_version}" YAMORY_IMAGE_NAME="ghcr.io/cybozu/${ubuntu_image}:${TAG}" bash -c "$(curl -sSf -L https://mw-receiver.yamory.io/image/script/trivy)"

Changed to execute a scan job when the build job is finished. #340

Changed to execute a scan job when the build job is finished. #340

Conversation

012xx commented Nov 15, 2024 • edited Loading

Overview

Why

What

Choose a reason for hiding this comment

takahiro-yamada left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

takahiro-yamada left a comment

Choose a reason for hiding this comment

012xx commented Nov 15, 2024 •

edited

Loading