Continuous Integration Workflow for Non-iOS

[dgor82]

Refer to [[2024-03-19 Paul call - CI vs. CD for Mobile]]

• Create Release solution configuration (in IDE!) for different targets

  • 1. All except iOS
  • 2. iOS Only

• CI

  • Use Ubuntu-latest instance in build_and_test workflow
  • Go through previous workflows to recreate basics like version tagging, create and merge, check for pr merge signal etc.
  • Install dotnet workloads 'wasi-experimental' and 'android'
  • Adapt dotnet restore and build commands for the 'no iOS' release configuration
  • Temp step for dev/debug only: AFTER (!) dotnet restore/build (because of apparent lazy installation of workloads), use run: find ~/.dotnet -type f > afterWorkloads.txt (and download as artefact) to figure out where in the runner's file system the workloads actually get saved
  • Then configure the caching path for the workloads accordingly, to prevent multi-GB workload downloads on every workflow run

[dgor82]

So most likely I won't need the above temp code to list files and output as artefacts because I now know, from observing on my mac, that on linux/unix systems this is where dotnet installs go:

1. most of dotnet sdk and installed workloads go to /usr/local/share/dotnet/ and most of the GBs from workloads into /packs/
2. a bit of Cache and sentinels and some Telemetry stuff goes into ~/.dotnet

Now, I'm assuming that on a ubuntu Github runner it will be the same folders and thus, the entire /dotnet/ folder should be cached for dotnet (with a hash of global.json as the key...).

In regards to caching nuget... hmm

1. ~/.nuget/ by default is the folder where locally any nuget packages are stored. Cache this, with a hash of all .csproj files! Since they contain version numbers of dependencies.
2. but doesn't 'dotnet restore' anyway download all nugets a new? Ah no, it probably checks if they are already there...

[dgor82]

From my eNotes:

• Caching
  • Does the build-int cache for dotnet setup action work without package.lock.json? If not, do manual. See also this.
  • Layers
    • . NET Basic Framework
    • Project specific NuGet / Dependencies
    • Workloads (dotnet install workloads seems to install most of the GBs into the /usr/local/share/dotnet/ folder! Maybe I simply need to just cache that one?! )
    • Android SDK
      • --> Different from android workload!!
      • Is it pre-installed on the github runner? If not, needs sep. step for installation
      • Whether pre-installed or not, MSBuild probably needs to be told where to find it via the AndroidSdkDirectory prop
  • Logs from re-installing dotnet on my mac
    • dotnet 8.0.203: 640 MB in /usr/local/share/dotnet
    • dotnet workload install ios -> ca. 2.4 GB into /usr/local/share/dotnet mostly into /packs/ (for a total of 2.95 GB now)
    • dotnet workload install android -> ca. 3.1 GB added to /dotnet/ folder which now is 6.11 GB. It apparently also added some stuff to ~/Library/Android/sdk/
    • dotnet workload install wasm-tools -> ca. 1 GB added to /dotnet/ now is 7.17 GB
• Deterministic builds
  • Locked Mode with packages.lock.json vs. .csproj and implicit
  • Automated testing reduces need?
• Separate CI/CD for iOS vs. Non-iOS
  • Based on Solution / Release Configurations
  • Use MSBuild props: dotnet restore /p:Configuration=xyz

[dgor82]

Idea re workflows, fb vs. main

• Why not use the local dev machine to run restore/build/test before pushing the fb/* branch, this way avoiding the double build on fb/ and again on main. The local build could be guaranteed by convention plus the local 'finish work' script. This will cut down github actions usage time by half and accelerate feedback from CI/CD.
• After merger with main, the restore/build/test cycle runs on main, which is important to test for any possible conflicts with mergers from other devs.

[dgor82]

Decided:

1. Not to use Caching for the Android SDK (see notes in parked-workflows.txt)

2. Not to use 'Locked Mode' and packages.lock.json with dotnet, for these reasons:

a) .csproj specifies the used dependencies
b) automated tests reduce the absolute need for deterministic builds
c) more frequent changes to the nuget packages doesn't hurt CI/CD much, even if caching can't be used, because the nuget packages are restored very quickly on GitHub runners
d) thus can avoid the annoying overhead with constantly changing package.lock.json files during development.