Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

go.mod FilePath ReplaceDirective fails copying folder. #1829

Open
jtszalay opened this issue Jun 20, 2024 · 5 comments · May be fixed by #1830
Open

go.mod FilePath ReplaceDirective fails copying folder. #1829

jtszalay opened this issue Jun 20, 2024 · 5 comments · May be fixed by #1830

Comments

@jtszalay
Copy link

What version of gazelle are you using?

0.37.0

What version of rules_go are you using?

0.48.0

What version of Bazel are you using?

7.1.2

Does this issue reproduce with the latest releases of all the above?

Yes.

What operating system and processor architecture are you using?

x86_64 Ubuntu

What did you do?

Trying to use go.mod FilePath ReplaceDirective with repo https://github.com/ngrok/ngrok-go

Our repo uses direnv and nix to setup the environment.

What did you expect to see?

I expected to be able to use the local replace.

(If I delete the .direnv folder from my local repo checkout it works.)

What did you see instead?

It fails.

INFO: Repository gazelle~~go_deps~com_ngrok_golang_ngrok instantiated at:
  <builtin>: in <toplevel>
Repository rule go_repository defined at:
  /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~/internal/go_repository.bzl:379:32: in <toplevel>
ERROR: An error occurred during the fetch of repository 'gazelle~~go_deps~com_ngrok_golang_ngrok':
   Traceback (most recent call last):
        File "/var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~/internal/go_repository.bzl", line 279, column 17, in _go_repository_impl
                fail(command)
Error in fail: ["/var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~non_module_deps~bazel_gazelle_go_repository_tools/bin/fetch_repo", "--path", "/home/james/ngrok-go", "--dest", /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~go_deps~com_ngrok_golang_ngrok]
ERROR: <builtin>: fetching go_repository rule //:gazelle~~go_deps~com_ngrok_golang_ngrok: Traceback (most recent call last):
        File "/var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~/internal/go_repository.bzl", line 279, column 17, in _go_repository_impl
                fail(command)
Error in fail: ["/var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~non_module_deps~bazel_gazelle_go_repository_tools/bin/fetch_repo", "--path", "/home/james/ngrok-go", "--dest", /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~go_deps~com_ngrok_golang_ngrok]
ERROR: no such package '@@gazelle~~go_deps~com_ngrok_golang_ngrok//policy': ["/var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~non_module_deps~bazel_gazelle_go_repository_tools/bin/fetch_repo", "--path", "/home/james/ngrok-go", "--dest", /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~go_deps~com_ngrok_golang_ngrok]
ERROR: /home/james/ngrok/go/cmd/ngrok/tung/BUILD.bazel:4:11: //go/cmd/ngrok/tung:tung depends on @@gazelle~~go_deps~com_ngrok_golang_ngrok//policy:policy in repository @@gazelle~~go_deps~com_ngrok_golang_ngrok which failed to fetch. no such package '@@gazelle~~go_deps~com_ngrok_golang_ngrok//policy': ["/var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~non_module_deps~bazel_gazelle_go_repository_tools/bin/fetch_repo", "--path", "/home/james/ngrok-go", "--dest", /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~go_deps~com_ngrok_golang_ngrok]
ERROR: Analysis of target '//go/cmd/ngrok/app:app' failed; build aborted: Analysis failed

Attempting to run the fetch_repo script manually:

└─> /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~non_module_deps~bazel_gazelle_go_repository_tools/bin/fetch_repo --path /home/james/ngrok-go --dest /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~go_deps~com_ngrok_golang_ngrok
fetch_repo: mkdir /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~go_deps~com_ngrok_golang_ngrok/.direnv: file exists

Deleting that dir I then get the following:

└─> /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~non_module_deps~bazel_gazelle_go_repository_tools/bin/fetch_repo --path /home/james/ngrok-go --dest /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~go_deps~com_ngrok_golang_ngrok
fetch_repo: write /var/lib/ngrok/bazel/ngrok-bazel/external/gazelle~~go_deps~com_ngrok_golang_ngrok/.direnv/flake-inputs/kk6d4ip6m8b11k9rm2fw2a03v919pi4v-source: copy_file_range: is a directory

Is there a way to tell it to ignore that .direnv directory. It doesn't need it as a dep.
@fmeum
Copy link
Member

fmeum commented Jun 20, 2024

It seems reasonable to skip all top-level directories that start with a .. Would you be interested in sending a PR?

@jtszalay
Copy link
Author

Yeah! I'm easily able to recreate it so I'll work on making a PR.

@jtszalay
Copy link
Author

jtszalay commented Jun 20, 2024 •
edited
Loading

Ah! I found what the true issue is. It doesn't handle things that are symlinks:

It seems reasonable to skip all top-level directories that start with a ..
I think this is still a fine solution however.

└─> ls -al ../ngrok-go/.direnv/flake-inputs/                                         
total 8
drwxr-xr-x 2 james ngrok 4096 Jun 20 12:26 .
drwxr-xr-x 3 james ngrok 4096 Jun 20 12:26 ..
lrwxrwxrwx 1 james ngrok   50 Jun 20 12:26 kk6d4ip6m8b11k9rm2fw2a03v919pi4v-source -> /nix/store/kk6d4ip6m8b11k9rm2fw2a03v919pi4v-source
lrwxrwxrwx 1 james ngrok   50 Jun 20 12:26 qkig73szmrhgp0qhncxy5vb36lw2g3jj-source -> /nix/store/qkig73szmrhgp0qhncxy5vb36lw2g3jj-source
lrwxrwxrwx 1 james ngrok   50 Jun 20 12:26 sl31dgnpwinzvzhh7ak2rx3h6w7kbb30-source -> /nix/store/sl31dgnpwinzvzhh7ak2rx3h6w7kbb30-source
lrwxrwxrwx 1 james ngrok   50 Jun 20 12:26 yj1wxm9hh8610iyzqnz75kvs6xl8j3my-source -> /nix/store/yj1wxm9hh8610iyzqnz75kvs6xl8j3my-source

Edit: excluding dirs that start with . causes hash mismatch on downloaded dependencies :/

		// Check if this is a top-level directory that starts with a `.`
		if info.IsDir() && filepath.Dir(rel) == "." && filepath.Base(rel)[0] == '.' {
			return filepath.SkipDir
		}

Error in fail: gazelle~~go_deps~org_golang_x_net: fetch_repo: resulting module with sum h1:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=; expected sum h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
ERROR: no such package '@@gazelle~~go_deps~org_golang_x_net//idna': gazelle~~go_deps~org_golang_x_net: fetch_repo: resulting module with sum h1:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=; expected sum h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
ERROR: /home/james/ngrok/go/cmd/ngrok/config/BUILD.bazel:4:11: //go/cmd/ngrok/config:config depends on @@gazelle~~go_deps~org_golang_x_net//idna:idna in repository @@gazelle~~go_deps~org_golang_x_net which failed to fetch. no such package '@@gazelle~~go_deps~org_golang_x_net//idna': gazelle~~go_deps~org_golang_x_net: fetch_repo: resulting module with sum h1:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=; expected sum h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=

@fmeum
Copy link
Member

fmeum commented Jun 20, 2024

Ah, that makes sense. Then let's try to fix symlinks instead :-)

jtszalay added a commit to jtszalay/bazel-gazelle that referenced this issue Jun 20, 2024
@jtszalay
Resolve issue with directory symlinks in local replace. bazel-contrib…
9be8f0b 
…#1829
jtszalay added a commit to jtszalay/bazel-gazelle that referenced this issue Jun 20, 2024
@jtszalay
Resolve issue with directory symlinks in local replace. bazel-contrib…
95773d1 
…#1829
@jtszalay jtszalay linked a pull request Jun 20, 2024 that will close this issue
Open
@jtszalay
Copy link
Author

jtszalay commented Jun 20, 2024 •
edited
Loading

#1830
With this our local replace is working with the .direnv directory present.
Ah but the directory internals aren't copied. It just stops. Perhaps we need a more robust solution akin to https://github.com/moby/moby/blob/master/daemon/graphdriver/copy/copy.go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Resolve issue with directory symlinks in local replace. #1829 jtszalay/bazel-gazelle
2 participants
@jtszalay @fmeum