Code Cache Directory Structure

[bylaws]

Was thinking through how cache could be structured on ARM64EC and came up with something along the lines of the following:

• %LOCALAPPDATA%/caches/RDR2.exe-7821398321/{RDR2.exe-7821398321,ntdll.dll-8732832,kernel32.dll-6832732872,metadata}
• %LOCALAPPDATA%/codemaps/{RDR2.exe-7821398321,SetupInst.exe-32188932}/{ntdll.dll-8732832,RDR2.exe-7821398321 <or> SetupInst.exe-32188932}
  FEX records codemaps to %LOCALAPPDATA%/codemaps/pending/{rand()} 

On startup: Parse and split pending codemaps, build index from filepath to fileid, purge all deleted or old codemaps by checking fileid for each on-disk file. For every main exe codemap folder,  read metadata for cache folder, if settings dont match regen all else only regen ones that don't exist (e.g. deleted due to update) or have some threshold of missing rip entries over the codemap

Also additionally need to know when to regenerate if codemaps have expanded, not sure how this is done for Linux, but some threshold in the number of entrypoints seems reasonable

[neobrain]

Does WoA require a per-folder filesystem layout? Linux side we adhere to XDG_CACHE_DIR for use cases that prefer a per-game type of deployment, whereas the out-of-the-box setup uses a unified folder structure where library code maps are shared among applications. Seems like that would work for WoA as well, though I'm not sure what the equivalent of XDG_CACHE_DIR would be (maybe inspiration can be taken from Mesa shader caches?).

FEX records codemaps to %LOCALAPPDATA%/codemaps/pending/{rand()}

The app name isn't included here, which is how I'd have done it, but was this intentional? Just curious what's different about pending code maps here.

Is there a reason to prefer rand() over <appname>-<fileid>? If cache generation is triggered manually it probably doesn't make a difference, but would be nice for consistency (and to ease human inspection when this folder is shared between apps).

Using a pending subdirectory is a good idea. Probably want to have the top-level codemaps folder contain just that and a ready folder for the combined codemaps.

On startup: Parse and split pending codemaps, build index from filepath to fileid, purge all deleted or old codemaps by checking fileid for each on-disk file. For every main exe codemap folder, read metadata for cache folder, if settings dont match regen all else only regen ones that don't exist (e.g. deleted due to update) or have some threshold of missing rip entries over the codemap

read metadata for cache folder,

What type of metadata did you have in mind? Linux-FEX stores all metadata (library dependencies + main executable main) in the merged code map. Would that work for WoA?

Also additionally need to know when to regenerate if codemaps have expanded, not sure how this is done for Linux, but some threshold in the number of entrypoints seems reasonable

The flow to reliably trigger cache regeneration (even without considering number of entrypoints) is rather intricate (esp with multiprocessing involved). I'm currently finishing up the follow-up to #5113, which will implement code map merging and automatic cache generation, so perhaps best to wait for that PR to see what's happening there.

The only properties checked for now are cache file existence, cache/codemap modification times, and FEX version (and then the important question is how to apply these to cache library dependencies).

[bylaws]

Does WoA require a per-folder filesystem layout? Linux side we adhere to XDG_CACHE_DIR for use cases that prefer a per-game type of deployment, whereas the out-of-the-box setup uses a unified folder structure where library code maps are shared among applications. Seems like that would work for WoA as well, though I'm not sure what the equivalent of XDG_CACHE_DIR would be (maybe inspiration can be taken from Mesa shader caches?).

LOCALAPPDATA is a shared folder between all apps, the windows equivalent of XDG_CACHE_DIR just within a prefix. Cache could be shared between prefixes using an XDG dir but due to using windows paths etc being used that isn't super simple and probably not much benefit. Most linux wine users just have a single prefix, and proton doesn't benefit from sharing across per-game prefixes as library caches are specific to a given game exe anyway.

FEX records codemaps to %LOCALAPPDATA%/codemaps/pending/{rand()}

The app name isn't included here, which is how I'd have done it, but was this intentional? Just curious what's different about pending code maps here.

Is there a reason to prefer rand() over <appname>-<fileid>? If cache generation is triggered manually it probably doesn't make a difference, but would be nice for consistency (and to ease human inspection when this folder is shared between apps).

If multiple instances of the same executable are running then rand() or similar allows both to be captured. Could prefix the name to it if desired.

Using a pending subdirectory is a good idea. Probably want to have the top-level codemaps folder contain just that and a ready folder for the combined codemaps.

On startup: Parse and split pending codemaps, build index from filepath to fileid, purge all deleted or old codemaps by checking fileid for each on-disk file. For every main exe codemap folder, read metadata for cache folder, if settings dont match regen all else only regen ones that don't exist (e.g. deleted due to update) or have some threshold of missing rip entries over the codemap

read metadata for cache folder,

What type of metadata did you have in mind? Linux-FEX stores all metadata (library dependencies + main executable main) in the merged code map. Would that work for WoA?

The FEX setting dump used for that compilation, things like if steamstub is used and perhaps entrypoint counts in a consistent format to avoid needing to open every file to determine if to regen.

Also additionally need to know when to regenerate if codemaps have expanded, not sure how this is done for Linux, but some threshold in the number of entrypoints seems reasonable

The flow to reliably trigger cache regeneration (even without considering number of entrypoints) is rather intricate (esp with multiprocessing involved). I'm currently finishing up the follow-up to #5113, which will implement code map merging and automatic cache generation, so perhaps best to wait for that PR to see what's happening there.

The only properties checked for now are cache file existence, cache/codemap modification times, and FEX version (and then the important question is how to apply these to cache library dependencies).

[bylaws]

Following up in #5127

It is somewhat awkward for windows to use an increasing index for pending codemaps and since all that are pending here will be scanned anyway, will adopt -TimeStamp.