CI: add benchmark workflow and script

[singh1203]

Fixes: #85

What changes are included in this PR?

Added Benchmark CI using Debian as runtime environment

Are these changes tested?

Yes, I have tested the changes locally.

[zeroshade]

Just a few nitpicks but we also need to add the CONBENCH_EMAIL and CONBENCH_PASS secrets to this repo

[singh1203]

Hey @zeroshade, it would be good if you could share your thoughts here.
Thank you!
cc - @assignUser

[kou]

How about running only ci/scripts/bench.sh --json on pull request (and push on fork too?) for testing?

- name: Run Benchmarks
  if: github.event_name == 'pull_request'
  run: |
    ci/scripts/bench.sh --json
- name: Run Benchamrks
  if: >-
    success() && github.event_name == 'push' && github.repository == 'apache/arrow-go' && github.ref_name == 'main'
  env:
    ...
  run: |
    ...

If it's possible, the following may be better:

- name: Run benchmarks
  run: |
    ci/scripts/bench.sh --json
- name: Upload results
  if: github.event_name == 'push' && github.repository == 'apache/arrow-go' && github.ref_name == 'main'
  env:
    ...
  run: |
    ...

[singh1203]

Well, I went with:

- name: Run benchmarks
  run: |
    ci/scripts/bench.sh --json
- name: Upload results
  if: github.event_name == 'push' && github.repository == 'apache/arrow-go' && github.ref_name == 'main'
  env:
    ...
  run: |
    ...

IMO it makes more sense to have a trigger push event on the main branch and the previous review from @assignUser here suggests that benchmark dumps are quite large and no pull request based triggering.

[zeroshade]

Hey @singh1203 ! I'm at FOSDEM25 this weekend, I'll try to give this a look but I might not get to it until Monday/Tuesday.

[singh1203]

@kou Please let me know if I missed something, guidance will be appreciated.
Thank you!

[kou]

If we have paths for push, this workflow isn't executed for most commits on main. Because most commits don't change these files.

How about using paths only for pull_request?

Suggested change

	push:
	branches: [main]
	paths:
	- ".github/workflows/benchmark.yml"
	- "ci/scripts/bench_adapt.py"
	push:
	branches: [main]
	pull_request:
	paths:
	- ".github/workflows/benchmark.yml"
	- "ci/scripts/bench_adapt.py"

If we have pull_request.paths, we can test this workflow only in PRs that change these files such as this PR.

[kou]

ci/scripts/bench_adapt.py will run ci/scripts/bench.sh. So we don't need this with push.

Suggested change

	- name: Run Benchmarks
	- name: Run Benchmarks
	if: github.event_name != 'push'