Where are the benchmarks posted?

[Red-Portal]

Hi, it seems like the benchmarks are no longer being posted on turinglang.org/AdvancedVI.jl/benchmark. Was it moved to a different url? @yebai @shravanngoswamii

[shravanngoswamii]

We changed the benchmark workflow to run only on PRs! See here: #74

[Red-Portal]

Then is there a way I can monitor change in performance across commits?

[shravanngoswamii]

Then is there a way I can monitor change in performance across commits?

As of now, I don't think there is any way! We have to roll back to previous workflow if that's required!

[yebai]

Since every (substantial) change has to go through a PR, it should be sufficient to benchmark PRs only. Even with the previous setup, it only benchmarks the final commit in PRs instead of all commits.

@shravanngoswamii What is strange is benchmarks seem not to be running even for PRs; see: #84

[Red-Portal]

I can imagine that performance bugs can appear and disappear due to changes upstream, not just local changes introduced through PRs. Benchmarking only on PRs won't be able to catch such performance bugs in a timely fashion would it?

[yebai]

Benchmarks are back live: https://turinglang.org/AdvancedVI.jl/benchmarks/

I don't think benchmarks are meant to track upstream performance regression. If you want to do that, you may want to set up a regular interval to run benchmarks, e.g. daily / weekly.