Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for releaseServerUrl #70

Merged
merged 19 commits into from
Sep 30, 2024
Merged

Add support for releaseServerUrl #70

merged 19 commits into from
Sep 30, 2024

Conversation

juxtin
Copy link
Contributor

@juxtin juxtin commented Sep 27, 2024

This is based on #67, with one additional commit to fix tests. Since @jhutchings1 has sadly moved on to other things, I've taken over this PR to see it through.

Original PR description from #67:

This PR adds a new optional field called releaseServerUrl. This sets the URL that you should use to query the microsoft/component-detection repository. In the only case I know of, if you set this, you're on GitHub Enterprise Server, and you aren't able to get the component-detection binary yourself. This means you'll grab the action from your local server, but the binary from github.com. The value should always be https://api.github.com.

I tested this on GHES 3.13. Fixes #42

@jhutchings1

That's the main part of the PR. The part that I added was using babel-jest to fix the tests. In short, tests were failing because Jest couldn't properly handle ES modules like octokit and @github/dependency-submission-toolkit, which are mixed with CommonJS modules in this project. Adding babel-jest allows Jest to transform those modules on the fly so that they can be used in tests. It has no effect on the actual code.

I also removed the configuration option because it only had one possible value and only in one specific case, which I think we can detect automatically. In this version, it should Just Work™ without any intervention.

Copy link

@Ahmed3lmallah Ahmed3lmallah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some minor comments, but LGTM otherwise.

componentDetection.ts Outdated Show resolved Hide resolved
action.yml Outdated
@@ -21,6 +21,9 @@ inputs:
detectorsFilter:
description: 'A comma separated list with the identifiers of the specific detectors to be used. This is meant to be used for testing purposes only.'
required: false
releaseServerUrl:
description: 'The baseUrl of the release server to use. If you set this, it should be set to `https://api.github.com`'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description is a bit confusing to me: Does it mean that https://api.github.com is the default value, in which case we should change the wording. I don't think it does now that I looked at the code.

Otherwise, if it means that https://api.github.com is the only possible valid value, then we should probably hardcode the url and use a boolean instead to trigger that option.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that is confusing. Let me figure that out and update.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I see what the original intent was and I rewrote it so that it should work without setting any configuration options. I'm going to try to test this in a real GHES instance to make sure.

Copy link

@bteng22 bteng22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM too! Pulled down the branch and ran the tests just in case.

I got this timeout error a couple times but after reloading the codespace I haven't encountered it again. Looks like it might've been an issue with my machine

  ● Downloads CLI

    thrown: "Exceeded timeout of 5000 ms for a test.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      10 |   await ComponentDetection.downloadLatestRelease();
      11 |   await ComponentDetection.runComponentDetection("./test");
    > 12 |   expect(fs.existsSync(ComponentDetection.outputPath));
         | ^
      13 | });
      14 |
      15 | test("Parses CLI output", async () => {

      at componentDetection.test.ts:12:1

@juxtin
Copy link
Contributor Author

juxtin commented Sep 28, 2024

Thanks @bteng22, I wonder if that was just a weird fluke. I haven't seen that in any of my test runs, nor has it been happening in CI.

I've now confirmed that this works in GHES 3.13 and on github.com, so I think it's ready to merge and release. I'll do that on Monday, though, to avoid tempting fate.

@juxtin juxtin merged commit d7bc998 into main Sep 30, 2024
4 checks passed
@nihalraghuraj-assent
Copy link

nihalraghuraj-assent commented Oct 31, 2024

Hi juxtin, I tried implementing the action against the latest version v0.0.3 on GHES 3.13
Still no dice, fails to create a snapshot for some reason.
Could you please help me diagnose this issue??

Following the release of the new version, v0.0.3 and the conversion here: Fixed-issue, I tried to implement the following on my end:

name: Dependency Submission

on:
  push:
    branches: [ master, develop ]
  pull_request:
    branches: [ develop ]
  workflow_dispatch:

jobs:
  Component-Detection-Dependency:
    runs-on: windows-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Execute restore
        run: dotnet restore ./Assent.Party.Api.sln      

      - name: Download component-detection.exe
        run: |
          $output = "${env:GITHUB_WORKSPACE}\component-detection.exe"
          if (Test-Path $output) {
            Write-Output "Component Detection file already exists at $output"
          } else {
            $url = "https://github.com/microsoft/component-detection/releases/download/v5.1.5/component-detection-win-x64.exe"
            Invoke-WebRequest -Uri $url -OutFile $output
            Write-Output "Component Detection file downloaded to $output"
          }

      - name: List files in the current directory and check if component-detection.exe exists
        run: |
          Get-ChildItem -Path "${env:GITHUB_WORKSPACE}" | Format-Table Name, Length, LastWriteTime
          $filePath = "${env:GITHUB_WORKSPACE}\component-detection.exe"
          if (Test-Path -Path $filePath) {
              Write-Output "Component Detection file exists"
          } else {
              Write-Output "Component Detection file does not exist"
              exit 1
          }

      - name: Component Detection dependency submission action
        uses: advanced-security/[email protected]

But still end up with the same error as below:

Run advanced-security/[email protected]
  
Running component-detection
C:\actions-runner-3\_work\usvc-party\usvc-party\component-detection.exe scan --SourceDirectory . --ManifestFile ./output.json
[[1](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:1)6:28:47 INF] Finding components...
[16:28:47 INF] Starting enumeration of C:\actions-runner-3\_work\usvc-party\usvc-party
[16:28:47 INF] No instructions received to scan docker images.
[16:28:47 INF] Enumerated 471 files and 63 directories in 00:00:00.0287646
[16:28:49 INF] No python found on system. Python detection will not run.
                               Detection Summary                                
┌───────────────────┬───────────────────┬───────────────────┬──────────────────┐
│ Component         │ Detection Time    │ # Components      │ # Explicitly     │
│ Detector Id       │                   │ Found             │ Referenced       │
├───────────────────┼───────────────────┼───────────────────┼──────────────────┤
│ CocoaPods         │ 0.1[5](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:5) seconds      │ 0                 │ 0                │
│ Go                │ 0.14 seconds      │ 0                 │ 0                │
│ Gradle            │ 0.14 seconds      │ 0                 │ 0                │
│ Ivy (Beta)        │ 0.16 seconds      │ 0                 │ 0                │
│ Linux             │ 0.043 seconds     │ 0                 │ 0                │
│ MvnCli            │ 0.16 seconds      │ 0                 │ 0                │
│ Npm               │ 0.14 seconds      │ 0                 │ 0                │
│ NpmLockfile3      │ 0.14 seconds      │ 0                 │ 0                │
│ NpmWithRoots      │ 0.14 seconds      │ 0                 │ 0                │
│ NuGet             │ 0.14 seconds      │ 0                 │ 0                │
│ NuGetPackagesConf │ 0.14 seconds      │ 0                 │ 0                │
│ ig                │                   │                   │                  │
│ NuGetProjectCentr │ 0.22 seconds      │ 192               │ 36               │
│ ic                │                   │                   │                  │
│ PipReport         │ 1.[5](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:6) seconds       │ 0                 │ 0                │
│ Pnpm              │ 0.15 seconds      │ 0                 │ 0                │
│ Poetry (Beta)     │ 0.1[6](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:7) seconds      │ 0                 │ 0                │
│ Ruby              │ 0.16 seconds      │ 0                 │ 0                │
│ RustCli           │ 0.16 seconds      │ 0                 │ 0                │
│ RustCrateDetector │ 0.16 seconds      │ 0                 │ 0                │
│ Vcpkg             │ 0.16 seconds      │ 0                 │ 0                │
│ Yarn              │ 0.16 seconds      │ 0                 │ 0                │
│ ───────────────── │ ───────────────── │ ───────────────── │ ──────────────── │
│ Total             │ 1.[7](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:8) seconds       │ 192               │ 36               │
└───────────────────┴───────────────────┴───────────────────┴──────────────────┘
[16:2[8](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:9):49 INF] 
[16:28:49 INF] 
[16:28:49 INF] Detection time: 1.6739984 seconds.
[16:28:4[9](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:10) INF] Scan Manifest file: C:\actions-runner-3\_work\usvc-party\usvc-party\output.json
Getting manifests from results
Notice: Submitting snapshot...
Notice: {
    "detector": {
        "name": "Component Detection",
        "version": "0.0.1",
        "url": "https://github.com/advanced-security/component-detection-dependency-submission-action"
    },
    "version": 0,
    "job": {
        "correlator": "Component-Detection-Dependency",
        "id": "12803"
    },
    "sha": "418ca47180282f31120390027abafb31bee9793a",
    "ref": "refs/pull/478/merge",
    "scanned": "2024-[10](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:11)-31T16:28:49.819Z",
    "manifests": {
        "/test/Assent.Party.Api.UnitTests/Assent.Party.Api.UnitTests.csproj": {
            "resolved": {
                "pkg:nuget/[email protected]": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuget/[email protected]": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuget/Microsoft.TestPlatform.ObjectModel@[17](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:18).1.0": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuget/[email protected]": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuget/[email protected]": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuget/[email protected]": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuget/[email protected]": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuget/[email protected]": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuget/Moq@4.[18](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:19).1": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": [
                        "pkg:nuget/[email protected]",
                        "pkg:nuget/[email protected]",
                        "pkg:nuget/[email protected]"
                    ]
                },
                "pkg:nuget/[email protected]": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuget/[email protected]": {
                    "package_url": "pkg:nuget/[email protected]",
                    "relationship": "indirect",
                    "scope": "runtime",
                    "dependencies": []
                },
                "pkg:nuge
Error: Cannot read properties of null (reading 'toString')
Error: TypeError: Cannot read properties of null (reading 'toString')
    at C:\actions-runner-3\_work\_actions\advanced-security\component-detection-dependency-submission-action\v0.0.3\webpack:\component-detection-action\node_modules\@github\dependency-submission-toolkit\dist\index.js:506:1
    at Generator.next (<anonymous>)
    at fulfilled (C:\actions-runner-3\_work\_actions\advanced-security\component-detection-dependency-submission-action\v0.0.3\webpack:\component-detection-action\node_modules\@github\dependency-submission-toolkit\dist\index.js:395:1)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

C:\actions-runner-3\_work\_actions\advanced-security\component-detection-dependency-submission-action\v0.0.3\webpack:\component-detection-action\node_modules\@github\dependency-submission-toolkit\dist\index.js:395
        function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
^
Error: Failed to submit snapshot: TypeError: Cannot read properties of null (reading 'toString')
    at C:\actions-runner-3\_work\_actions\advanced-security\component-detection-dependency-submission-action\v0.0.3\webpack:\component-detection-action\node_modules\@github\dependency-submission-toolkit\dist\index.js:5[20](https://git.assentcompliance.com/assent-compliance/usvc-party/actions/runs/12803/job/41047?pr=478#step:6:21):1
    at Generator.next (<anonymous>)
    at fulfilled (C:\actions-runner-3\_work\_actions\advanced-security\component-detection-dependency-submission-action\v0.0.3\webpack:\component-detection-action\node_modules\@github\dependency-submission-toolkit\dist\index.js:395:1)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
   
    ```
    Could you please help me understand, what I might be missing here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add GitHub Enterprise Server support
5 participants