Android CI: Fix concurrent android sdk download

[vaslabs]

• Fixes parallel app testing conflicts on sdkmanager image download and conflicting on emulators.
• I've tried two approaches, one with syncrhonized and one with file lock. The synchronized approach was not consistently working but file lock seems to work with share.

I've rarely used file locks, so please check and let me know if I need to fix something or if there's a better alternative (mill utility?) to use.

Fixes #6522

[vaslabs]

if this works out, we probably should create some sort of AndroidSdkManagerWorker that handles the installations for all modules (i.e. AndroidSdkModule) . I suspect the issue would exist in other parts but it's probably more stable due to the license check and the ndk is only used in one place in the CI

[vaslabs]

hm, need to dig deeper

9307] Warning: Observed package id 'tools' in inconsistent location '/home/runner/.android/sdk/tools-2' (Expected '/home/runner/.android/sdk/tools')
9307] Warning: Already observed package id 'tools' in '/home/runner/.android/sdk/tools'. Skipping duplicate at '/home/runner/.android/sdk/tools-2'
9307] Warning: Observed package id 'tools' in inconsistent location '/home/runner/.android/sdk/tools-2' (Expected '/home/runner/.android/sdk/tools')
9307] Warning: Already observed package id 'tools' in '/home/runner/.android/sdk/tools'. Skipping duplicate at '/home/runner/.android/sdk/tools-2'
9307] Warning: Observed package id 'tools' in inconsistent location '/home/runner/.android/sdk/tools-2' (Expected '/home/runner/.android/sdk/tools')
9307] Warning: Already observed package id 'tools' in '/home/runner/.android/sdk/tools'. Skipping duplicate at '/home/runner/.android/sdk/tools-2'

[vaslabs]

hm, something else is (also?) wrong, for some reason, the downloads get interrupted

[vaslabs]

thanks @souvlakias

[lefou]

I'm not convinced we want a general purpose generic action runner in the context of the worker. Can we instead move the download jobs into the worker and use them via API? The worker then can do whatever is needed to guarantee it's working well.

[vaslabs]

I'm not convinced we want a general purpose generic action runner in the context of the worker. Can we instead move the download jobs into the worker and use them via API? The worker then can do whatever is needed to guarantee it's working well.

yep, will do it like that, the general purpose was just the quickest way to get there, but I can make the general purpose function private and expose domain methods + get the os calls in, effectively move some of the AndroidSdkModule methods to the worker

[lefou]

I thought, the AndroidSdkModule is already a singleton and the download happens in a single task, such that there don't happen concurrent downloads to the same location. So either my mental module isn't right, or there are some tasks where we didn't get it right, yet. Or, we manually use alternative AndroidSdkModules pointing to the same target location, and more then one of them.

[vaslabs]

I thought, the AndroidSdkModule is already a singleton and the download happens in a single task, such that there don't happen concurrent downloads to the same location. So either my mental module isn't right, or there are some tasks where we didn't get it right, yet. Or, we manually use alternative AndroidSdkModules pointing to the same target location, and more then one of them.

yeah, something is weird there, the synchronized didn't seem to work, even if I replaced the file lock with one of them. I think the AndroidSdkModule is not a singleton, it's extended in every example. But even so, I'd expect the private objects could be used as locks, but apparently not.

(Even so the presence of 3 locks could cause a deadlock, which might be one of the reasons the job occasionally times out)

[lefou]

So, we need to use a single ExternalModule to hold all the download logic (or if you want to use locks, to own the locks). If not we can't ensure the locks are singletons. Even some clever worker logic won't help, if the worker has multiple instances due to being bound to non-singleton AndroidSdkModules.

[vaslabs]

So, we need to use a single ExternalModule to hold all the download logic (or if you want to use locks, to own the locks). If not we can't ensure the locks are singletons. Even some clever worker logic won't help, if the worker has multiple instances due to being bound to non-singleton AndroidSdkModules.

Ok, I'll give it a go with the external module in the weekend, I was gonna do major work on the file lock approach anyway in the worker, might as well do the external module approach