[FileLocksmith]Move hanging operations to a distinct thread

[jaimecbernardo]

Summary of the Pull Request

File Locksmith is hanging for some users. After analyzing some dumps, we've detected there are some hanging system calls, especially NtQueryObject and GetFileType. It's not clear in what conditions this happens, so the work was offloaded to a different thread, which is destroyed and resumed if getting the information for a specific handle takes too long.
Users in the issues have confirmed this solves the problem for them.

PR Checklist

• Closes: File Locksmith apparently hanging #22203 and [File Locksmith] just displays busy circle #21621
• Communication: I've discussed this with core contributors already. If work hasn't been agreed, this work might be rejected
• Tests: Added/updated and all pass
• Localization: All end user facing strings can be localized
• Dev docs: Added/updated
• New binaries: Added on the required places
  • JSON for signing for new binaries
  • WXS for installer for new binaries and localization folder
  • YML for CI pipeline for new test projects
  • YML for signed pipeline
• Documentation updated: If checked, please file a pull request on our docs repo and link it here: #xxx

Detailed Description of the Pull Request / Additional comments

NtQueryObject and GetFileType are hanging for some handles.
This PR offloads the faulty code to a distinct thread, which is killed on a timeout and the work is resumed with a new thread.
Unfortunately, there are no alternative APIs that provide a timeout, meaning it's not possible to recover when these calls hang.
The solution presented here destroys the hanging threads, which is not advisable, but unfortunately there doesn't seem to be an alternative here except by offloading to a new process and destroying that process, which would complicate the code too much in my opinion.
There's also an additional condition before calling GetFileType so that it's only called on handles that make sense, to safeguard wrong calls.
Some additional tweaks to remove some warnings on other parts of the code. (Got those as errors on my tries to upgrade this project to only be /CLR on some specific files)

I've tried changing the project to only use /CLR on the needed interface code files so I could use newer C++ synchronization headers. I've tried using futures with a timeout but those need the thread to join at the end, and that's not possible with hanging calls. I've tried using packaged_tasks with a different thread for each handle, but that makes it so that the operation takes several minutes instead of seconds (millions of handles exist). I ended up reverting these tries.

Validation Steps Performed

Verified FileLocksmith still works well in this build. Had users confirm with a build that it solves the issue for them.

[lb90]

As noted in the comments, TerminateThread is unsafe and may leave a few mutexes in the abandoned state.

I wonder if CancelSynchronousIo may help with unlocking the thread? According to https://community.osr.com/discussion/176861/handle-info-using-ntquerysysteminformation-how-to-differentiate-files-from-folders GetFileType / NtQueryObject is blocked in a read call.

[PolarGoose]

@lb90 ,

I have tried to call NtQueryObject(..., ObjectNameInformation, ...) from a separate thread and call CancelSynchronousIo if it hangs. Unfortunately, it doesn't work. The CancelSynchronousIo doesn't unlock the NtQueryObject call.

I have found a very similar bug report from Python psutil library: get_open_files might hang forever #340
This bug report contains a lot of information. They have tried a lot of workarounds, and in the end, they also ended up killing the thread.

[lb90]

Ok, thank you very much!

[PolarGoose]

There is a proper solution to prevent hangs associated with the NtQueryObject(OBJECT_INFORMATION_CLASS.ObjectNameInformation) function.
The ReOpenFile can be used to exclude all non-file-related handles that might cause NtQueryObject to hang.
This approach was suggested by @dmex in this GitHub discussion.

I have successfully implemented this workaround in my project: ShowWhatProcessLocksFile
I confirm that it works very well and eliminates hanging issues.