Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Github] Add new workflow to build metrics container #117462

Merged
merged 8 commits into from
Nov 29, 2024

Conversation

boomanaiden154
Copy link
Contributor

This patch adds a new Github Actions workflow to build the metrics container and push it to the Github Container Registry.

This patch includes the script that pulls information from Github and pushes
it to Grafana. This is currently running in the cluster and pushes information
to https://llvm.grafana.net/public-dashboards/6a1c1969b6794e0a8ee5d494c72ce2cd.
This script is designed to accept other jobs relatively easily and can be
easily modified to look at other metrics.
This patch adds a new Github Actions workflow to build the metrics container
and push it to the Github Container Registry.
@boomanaiden154 boomanaiden154 marked this pull request as ready for review November 24, 2024 05:42
@llvmbot
Copy link
Member

llvmbot commented Nov 24, 2024

@llvm/pr-subscribers-github-workflow

Author: Aiden Grossman (boomanaiden154)

Changes

This patch adds a new Github Actions workflow to build the metrics container and push it to the Github Container Registry.


Full diff: https://github.com/llvm/llvm-project/pull/117462.diff

1 Files Affected:

  • (added) .github/workflows/build-metrics-container.yml (+78)
diff --git a/.github/workflows/build-metrics-container.yml b/.github/workflows/build-metrics-container.yml
new file mode 100644
index 00000000000000..2b73dc87ea9f82
--- /dev/null
+++ b/.github/workflows/build-metrics-container.yml
@@ -0,0 +1,78 @@
+name: Build Metrics Container
+
+permissions:
+  contents: read
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - .github/workflows/build-metrics-container.yml
+      - '.ci/metrics/**'
+  pull_request:
+    branches:
+      - main
+    paths:
+      - .github/workflows/build-metrics-container.yml
+      - '.ci/metrics/**'
+
+jobs:
+  build-metrics-container:
+    if: github.repository_owner == 'llvm'
+    runs-on: ubuntu-latest
+    outputs:
+      container-name: ${{ steps.vars.outputs.container-name }}
+      container-name-tag: ${{ steps.vars.outputs.container-name-tag }}
+      container-filename: ${{ steps.vars.outputs.container-filename }}
+  steps:
+    - name: Checkout LLVM
+      uses: actions/checkout@v4
+      with:
+        sparse-checkout: .ci/metrics/
+    - name: Write Variables
+      id: vars
+      run: |
+        tag=`date +%s`
+        container_name="ghcr.io/$GITHUB_REPOSITORY_OWNER/metrics"
+        echo "container-name=$container-name" >> $GITHUB_OUTPUT
+        echo "container-name-tag=$container_name:$tag" >> $GITHUB_OUTPUT
+        echo "container-filename=$(echo $container_name:$tag  | sed -e 's/\//-/g' -e 's/:/-/g').tar" >> $GITHUB_OUTPUT
+    - name: Build Container
+      working-directory: ./.ci/metrics
+      run: |
+        podman build -t ${{ steps.vars.outputs.container-name-tag }} -f Dockerfile .
+    # Save the container so we have it in case the push fails.  This also
+    # allows us to separate the push step into a different job so we can
+    # maintain minimal permissions while building the container.
+    - name: Save Container Image
+      run: |
+        podman save  ${{ steps.vars.outputs.container-name-tag }} >  ${{ steps.vars.outputs.container-filename }}
+    - name: Upload Container Image
+      uses: actions/upload-artifact@v4
+      with:
+        name: container
+        path: ${{ steps.vars.outputs.container-filename }}
+        retention-days: 14
+  
+  push-metrics-container:
+    if: github.event_name == 'push'
+    needs:
+      - build-metrics-container
+    permissions:
+      packages: write
+    runs-on: ubuntu-24.04
+    env:
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+    steps:
+      - name: Download Container
+        uses: actions/download-artifact@v4
+        with:
+          name: container
+      - name: Push Container
+        run: |
+          podman load -i ${{ needs.build-metrics-container.outptus.container-filename }}
+          podman tag ${{ needs.build-metrics-container.outputs.container-name-tag }} ${{ sneeds.build-metrics-container.outputs.container-name }}:latest
+          podman login -u ${{ github.actor }} -p $GITHUB_TOKEN ghcr.io
+          podman push ${{ needs.build-metrics-container.outputs.container-name-tag }}
+          podman push ${{ needs.build-metrics-container.outputs.container-name }}:latest

Copy link
Collaborator

@tstellar tstellar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow file looks fine to me. I'm not sure what this container is meant to do though.

@boomanaiden154
Copy link
Contributor Author

The workflow file looks fine to me. I'm not sure what this container is meant to do though.

It builds the container in #117461. I definitely could've made this a bit more clear in the commit description. Sorry about that.

It essentially packages the script in that PR which every five minutes grabs information about the workflows that have completed since the previous check and dumps the data into Grafana for visualization/alerting.

Copy link
Contributor

@Keenuts Keenuts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry thought I had approved this one earlier.

Base automatically changed from users/boomanaiden154/metrics-python-script-deps to main November 29, 2024 19:15
@boomanaiden154 boomanaiden154 merged commit 9ebab70 into main Nov 29, 2024
8 checks passed
@boomanaiden154 boomanaiden154 deleted the users/boomanaiden154/build-metrics-container branch November 29, 2024 19:35
TIFitis pushed a commit to TIFitis/llvm-project that referenced this pull request Dec 18, 2024
This patch adds a new Github Actions workflow to build the metrics
container and push it to the Github Container Registry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants