update-repos: add -parse_only flag

[sluongng]

Based on #1838 so please review that first.

This PR adds a -parse_only flag to gazelle update-repos which will strictly generate the go_repository repos definition by parsing the go.mod and go.sum file.

There are a few benefits to this approach:

1. Unlike the previous approach which relies on go list and go download, this approach requires no network call and thus, fast

2. If the go.mod file is generated correctly by Go tooling, the content of it should be subjected to Module graph pruning and does not include the transitive dependencies that are not needed for the build to work. As a result, we will generate fewer go_repository repos, sometimes by 6-10x compared to the previous approach. For this to work correctly, we heavily rely on user calling go mod tidy to do all the necessary network calls and edit go.mod and go.sum files.

3. This brings the current update-repos WORKSPACE setup to be on par with the BzlMod setup, which also parse go.mod and go.sum to generate go_repository repos. Instead of implementing rules_go own parsers, like the Starlark implementation, this PR tries to reuse code from x/mod and x/tools as much as possible. This made x/tools a direct dependency of Gazelle.

Similar to #1838, this PR is divided into smaller commits for easier reviewing.

[sluongng]

@fmeum this is failing with

(19:31:14) ERROR: error loading package '@@io_bazel_rules_go//proto': at /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/external/io_bazel_rules_go/proto/compiler.bzl:20:5: cannot load '@@rules_proto//proto:proto_common.bzl': no such file

which seems to be a rules_go issue? rules_proto needs to be part of go_rules_dependencies? 🤔

[fmeum]

rules_go only depends on rules_proto if you use proto functionality, which is why it was decided at some point to not provide this dep implicitly. You may have to update to 6.0.0 manually.

[sluongng]

I think that's a bit strange to make a non-dev dep "optional".
The deps are usually declared with maybe and will only be lazily downloaded when used.

So I don't see a reason why rules_proto needed to be excluded here.

[fmeum]

Adding it to the macro doesn't really help that much as the WORKSPACE setup for rules_proto requires the user to call three dep macros, which we can't do for them: https://github.com/bazelbuild/rules_proto/releases/tag/6.0.2

I would thus prefer to just leave this as is as the real solution is a (partial) migration to Bzlmod.

[tyler-french]

@sluongng Could we separate out all of the dependency changes to a different PR which we merge first?

This is a bit difficult to review, and might cause unexpected downstream behavior.

I would also like to avoid making any upgrades in go.mod file that aren't absolutely necessary.

[tyler-french]

@sluongng Could we separate out all of the dependency changes to a different PR which we merge first?

This is a bit difficult to review, and might cause unexpected downstream behavior.

I would also like to avoid making any upgrades in go.mod file that aren't absolutely necessary.

Oh, I just saw #1838, let me take a look there. It seems like there's some CI issues

[sluongng]

Yeah I have separated it to #1838. Let's discuss it there.