xmake update dev fails when not ptrace-d on GitHub CI

[nwf]

Xmake Version

v2.9.7+dev.686b414

Operating System Version and Architecture

GitHub CI ubuntu-22.04

Describe Bug

xmake update dev completes its own installation when run under strace (and so is being ptrace()-d), but does not when not being carefully observed, omitting several files, apparently at random, from ~/.local in the latter case.

I have no idea what's going on, and I would not believe me if I were telling me this. However, I'm not making this up, and the effect is consistent across a large number of runs tried so far. The only difference between https://github.com/nwf/cheriot-rtos/actions/runs/12575210568 (failure) and https://github.com/nwf/cheriot-rtos/actions/runs/12575263084 (success) is the absence (resp. presence) of strace -e file -f -o /dev/null on line 35 of the failing workflow file (resp. the successful one).

(This was originally found using the CHERIoT devcontainer in CI; see https://github.com/nwf/cheriot-rtos/actions/runs/12575072524 (success) and https://github.com/nwf/cheriot-rtos/actions/runs/12575106471 (failure), where again the difference is just a strace -f -o /dev/null on line 31 of the workflow file).)

Expected Behavior

xmake update dev should install all of its files

Project Configuration

n/a

Additional Information and Error Logs

Please see logs of the referenced CI jobs. I am unable to reproduce this failure locally.

[nwf]

Similar failure mode on ubuntu-24.04:

• Failure: https://github.com/nwf/cheriot-rtos/actions/runs/12575411195
• Success: https://github.com/nwf/cheriot-rtos/actions/runs/12575400550

[waruqi]

Why do not use https://github.com/xmake-io/github-action-setup-xmake on ci?

https://github.com/xmake-io/xmake-repo/blob/d5c975b507239791bbbf6882ffb66cbc44911804/.github/workflows/archlinux.yml#L30-L34

      - uses: xmake-io/github-action-setup-xmake@v1
        with:
          xmake-version: branch@dev

Please do not call xmake update on ci.

xmake update will call the xmake subprocess in the background to update itself. There will be a delay of 1-2 seconds, which does not block the update.

[nwf]

I did not know about that action, so thank you. It is, unfortunately, GitHub specific, and we use other CI engines as well. As such, concerning

xmake update will call the xmake subprocess in the background to update itself

the general rule of thumb for UNIX commands, as opposed to daemons, is that when they return to the shell, all processing has finished. Would you consider making that the default or at least a --synchronous flag or somesuch to optionally wait for background processing to finish?

(Failing that, I'm tempted to appeal to something like Linux's prctl SUBREAPER functionality and I'd really rather not have to do that... ETA: oh, hah, daemontools includes fghack for exactly this kind of thing... >_>)

[waruqi]

the general rule of thumb for UNIX commands, as opposed to daemons, is that when they return to the shell, all processing has finished. Would you consider making that the default or at least a --synchronous flag or somesuch to optionally wait for background processing to finish?

It needs to replace the xmake binary itself, so xmake needs to exit first.

you can call sleep 1 to wait some time after xmake update

[waruqi]

In addition, your problem should not be related to it, because the previous xmake was installed through apt, which is installed in the /usr directory, and it cannot support xmake update.

Only xmake installed through curl -fsSL https://xmake.io/shget.text | bash can well support xmake update

If you use apt/xmake to call xmake update dev, you need to run source ~/.xmake/profile to switch to local/xmake .

Otherwise apt/xmake will still be used.

xmake update dev
source ~/.xmake/profile

Or you can refer https://github.com/waruqi/dockerfiles/blob/master/ubuntu/Dockerfile , if you are using docker

[nwf]

I have worked around this issue in our GH CI with fghack; see https://github.com/CHERIoT-Platform/cheriot-rtos/pull/396/files . I believe the same workaround will suffice on our other CI platforms, at least for the moment. It's fragile and sad, but it works for the moment (until someone calls closeall); there doesn't seem to be a conveniently packaged prctl SUBREAPER utility in Ubuntu at the moment.

It needs to replace the xmake binary itself, so xmake needs to exit first.

That's not necessarily true. While ETXTBUSY is a thing, the traditional workaround is to create a new file and rename it into place. There is some risk, I suppose, of having write access to the file but not its containing directory, but that seems very remote.

In addition, your problem should not be related to it, because the previous xmake was installed through apt,

Yes, especially in the case of writing entirely new files to entirely new directories, there's no need for xmake to do this dance.

If you use apt/xmake to call xmake update dev, you need to run source ~/.xmake/profile to switch to local/xmake .

I understand that, that's unrelated to the failure observed, and we made sure to prune the apt-installed xmake in our CI runs to ensure that we were using the new binary. Please see the PR above for more detail.

[waruqi]

I have worked around this issue in our GH CI with fghack; see https://github.com/CHERIoT-Platform/cheriot-rtos/pull/396/files . I believe the same workaround will suffice on our other CI platforms, at least for the moment. It's fragile and sad, but it works for the moment (until someone calls closeall); there doesn't seem to be a conveniently packaged prctl SUBREAPER utility in Ubuntu at the moment.

I did not know about that action, so thank you. It is, unfortunately, GitHub specific, and we use other CI engines as well. As such, concerning

At least for github ci, you can use it to simplify your ci configuration.