Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1/3] Proper path resolution: add openat functionality #2254

Closed
wants to merge 8 commits into from

Conversation

yagehu
Copy link
Contributor

@yagehu yagehu commented Jun 14, 2024

This PR is the first in a series that will implement the path resolution algorithm described by wasi-filesystem. Since this new path resolution implementation requires the use of openat, this PR first introduces openat(2) and its Windows equivalent NtCreateFile to Wazero.

  • Linux implementation with openat(2)
  • Darwin implementation with trampoline to openat(2)
  • Windows implementation with NtCreateFile
  • Add more test cases to internal/sysfs/dirfs_test.go

The next PR will stack on this one and use the openat functionality to implement path resolution.

@yagehu yagehu force-pushed the path-res/1/openat branch 6 times, most recently from c4eb9c5 to 63aac67 Compare June 15, 2024 03:18
experimental/sys/fs.go Outdated Show resolved Hide resolved
@yagehu yagehu force-pushed the path-res/1/openat branch 2 times, most recently from 26f2ef6 to 6debef3 Compare June 17, 2024 16:12
This commit adds the `openat` functionality to `sys.FS`.  On UNIX-like
platforms, this is done with the `openat(2)` syscall.  On Windows, this
is done with [`NtCreateFile`][1].  This is the first in a series of
commit that overhauls how paths are resolved.  The wasi-filesystem spec
now [describes][2] a path resolution implementation that uses `openat`.

[1]: https://learn.microsoft.com/en-us/windows/win32/api/winternl/nf-winternl-ntcreatefile
[2]: https://github.com/WebAssembly/wasi-filesystem/blob/main/path-resolution.md

Signed-off-by: Yage Hu <[email protected]>
@yagehu yagehu force-pushed the path-res/1/openat branch from 6debef3 to ea28b4d Compare June 17, 2024 16:21
yagehu added 2 commits June 17, 2024 15:33
Signed-off-by: Yage Hu <[email protected]>
Signed-off-by: Yage Hu <[email protected]>
@yagehu yagehu force-pushed the path-res/1/openat branch from 1c64d11 to 107be03 Compare June 17, 2024 22:34
@yagehu yagehu force-pushed the path-res/1/openat branch 3 times, most recently from a960173 to 7f90143 Compare June 17, 2024 22:52
@yagehu yagehu force-pushed the path-res/1/openat branch from 7f90143 to 2dd5cda Compare June 17, 2024 23:05
@yagehu yagehu force-pushed the path-res/1/openat branch 2 times, most recently from 796b45e to 8b8f220 Compare June 21, 2024 04:58
@yagehu yagehu force-pushed the path-res/1/openat branch from 8b8f220 to 00f22bb Compare June 21, 2024 05:22
@yagehu yagehu changed the title [1/2] Proper path resolution: add openat functionality [1/3] Proper path resolution: add openat functionality Jun 21, 2024
@yagehu yagehu force-pushed the path-res/1/openat branch from 4071412 to 00f22bb Compare June 21, 2024 22:09
@yagehu yagehu marked this pull request as ready for review June 24, 2024 18:36
@yagehu yagehu requested a review from mathetake as a code owner June 24, 2024 18:36
@yagehu yagehu requested a review from evacchi June 24, 2024 18:36
@yagehu
Copy link
Contributor Author

yagehu commented Jun 24, 2024

FYI I can't stack PRs on top of this one because I can't push branches to this repo.

@@ -71,7 +72,7 @@ func (r *ReadFS) Utimens(path string, atim, mtim int64) experimentalsys.Errno {
}

// compile-time check to ensure readFile implements api.File.
var _ experimentalsys.File = (*readFile)(nil)
var _ fsapi.File = (*readFile)(nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was intentionally implementing only the experimental public VFS API. In fact I think this experimental VFS should not expose OpenAt

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context! I had thought it was an oversight.

Comment on lines +85 to +90
OpenAt(
fs experimentalsys.FS,
path string,
flag experimentalsys.Oflag,
mode fs.FileMode,
) (experimentalsys.File, experimentalsys.Errno)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need to expose this at this level?

the reason I am asking is that this obviously only applies to hierarchical FSs, but it won't work (as you can see later) on other types of file descriptors such as sockets or stdin/err/out. Could it be possible to keep this internal and limited to tree-like file systems?

Essentially, the final goal of this (series of) PR is to introduce an API that will reproduce the correct behavior when resolving paths, so it would great if we can scope it only to files for which "paths" make sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It definitely doesn't need to be exposed at this level. I misunderstood your last comment and thought you wanted the method moved from the public interface to a private one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, that was a fair assumption, but I was wondering if we could keep it as an implementation detail of Open for those FSs that need hierarchical file access 🤔

@evacchi evacchi self-assigned this Jun 25, 2024
@yagehu yagehu marked this pull request as draft June 25, 2024 18:01
@yagehu
Copy link
Contributor Author

yagehu commented Jul 28, 2024

Quick update: I'm on a cross-country move at the moment. Will get around to this again after mid-August. Thanks for the speedy reviews @evacchi

@mathetake
Copy link
Member

feel free to reopen when ready to continue the work

@mathetake mathetake closed this Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants