android: Added android_usecase kwargs to executable

[sp1ritCS]

This is the idea I've came up with in regards to #13758

It doesn't do anything regarding application glue right now, as I'm unsure if and how to implement it, but I thought about it some more and belive having it would simplify things quite a bit.

---

By setting android_usecase to application, the executable gets actually built as a shared library instead of an executable. This makes it possible to use an application within an android application process.

https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/7555/

[jpakkane]

Could you not already do this with the build_target function?

[sp1ritCS]

Perhaps. However my goal is to have a builder tool that you can just point at a meson project to create an apk from.

While you still have to modify your meson sources and to add the android_usecase, it seems less invasive than doing having users switch their excecutables to build_target(target_type: host_machine.os() == android ? 'shared_module' : 'executable', ...), especially given that they will have to add the android glue dependency too (declaring a link_whole dependency with it and adding it does seem to work within meson, not sure why cmake didn't like it)

What about adding a excecutable function to the gnome module that has the same prototype as a normal executable but that creates a library linked against the glue dependency if host_machine == 'android'?

[jpakkane]

Since you seem to have knowledge about this could you answer some questions I have about this?

How are Android applications actually done? FWICT it used to be the case that you could not have actual executables, instead it needed to be a shared library which you then opened from the Java side with whatever the equivalent of dlopen is. Can you now have actual executables? Can there be several? Can you invoke many processes within your sandbox? Is it possible (and if yes, do you intend to use) to make an Android application that is just native code without any Java/JVM?

[sp1ritCS]

It was always possible to have actual 'executables' on Android. Its just Linux after all and /system/bin/ contains a bunch of them the system relies on to work.

The thing is that users usually don't interact with them directly (outside of developers using adb shell) and executables do not have the possibility of interacting with the Android system (unless they use private non-stable interfaces), it is however possible for Applications to do the pipe+fork+exec combo to execute their own executables as child process. Doing so is discouraged however, with Google increasingly limiting its possibilities (phantom process killer, SELinux rules that enfore R^X for exec). Termux however makes heavy use of fork+exec to provide a POSIX like shell experience and is relatively popular as it allows you to use full SSH/tmux/vim/... on Android.

Is it possible (and if yes, do you intend to use) to make an Android application that is just native code without any Java/JVM?

No, you can make executables not applications. Applications are still special (Zygote) and need Java (ART) to interact with the stable OS interfaces provided.

My work of porting GTK to Android proper, needs an "Application" to work with and essentially works by having glue expose the proper JNI symbol and then calls main() from the actual codebase in a new thread.

[bruchar1]

Why don't you simply add a user option or a property to the cross file, and use it for the target type?

[sp1ritCS]

Why don't you simply add a user option or a property to the cross file, and use it for the target type?

My idea was need as little changes as possible to the meson files of existing applications and this was just my initial attempt. A better (but gtk specific solution) is proposed in #13800. It cuts down the currently needed build_target(host_machine.system() == 'android' ? 'shared_library' : 'executable', '...', dependencies: [...] + host_machine.system() == 'android' ? dependency('gtk4-android-glue') : [], ...) to import('gnome').executable(..., dependencies: [...], ...)

[jpakkane]

So, basically, the actual thing that you need to get is "build this target as an executable on all other platforms, but as a shared library in Android"? Are there any other use cases that need to be covered?

If true, then this could be implemented in one of several ways:

• tell people to use build_target and set the target type manually
• add some sort of module, custom function, etc that does the above
• add this (or some similar) kwarg to handle the issue
• something else, what?

The kwarg in this MR is similar to how win_subsystem works which is good for consistency.

[sp1ritCS]

Are there any other use cases that need to be covered?

There should be some mechanism that link-wholes a glue library (that provides the JNI entry function symbol) into the shared library. The idea I had originally for this MR was to have the ability to specify the dependency name in the cross file, but I didn't really look into how to implement it specifically. Obv. the alternative GNOME/GTK specific PR doesn't need that as it can just lookup the gtk4-android-glue dependency directly (for now).

The kwarg in this MR is similar to how win_subsystem works which is good for consistency.

That was the idea behind it :)

[jpakkane]

I'm not very fond of the name android_usecase. It is not very descriptive, though I have problems trying to come up with a better name (as is usually the case). android_exe is similarly poor. android_executable_type is too long. Does android_exe_type sound good for people?

[sp1ritCS]

Sure, android_exe_type is fine too.

About the issue of needing to include glue code: I've taken a closer look at how QT does it, and it looks like they initialize themselves in the JNI_OnLoad of the libqt and the libqtandroid plugin and to launch the actual user code they require the shared library name containing the application main() to be specified in the ApplicationManifest.xml with a <meta-data> tag. They then load the application shared library using dlopen/dlsym to run the main function. This way no glue code is needed on the application side.

I think I should be able to implement a similar mechanism for GTK, which means the the changes of this MR would be enough to make it acceptably work. The downsides of this however would be:

• Less logical ordering: GTK is going to be responsible to be loading the application which itself is going to use GTK.
• Using dlopen/dlsym is going to be more error prone instead of letting the linker do it for you (and any issues will only be detected at runtime)
• Unlike statically linking the glue code will call main into the application, it must be made sure that the main symbol will actually be exported, which isn't the case by default if people have add_project_arguments('-fvisibility=hidden', ...]). (This should be able to be remedied by modifying the cross file to add a linker script, meson should probably add that linker script in that case)
• It forces GTK to be a shared library (due to the required JNI_OnLoad symbol). But GTK currently only works as a shared library anyways, so this may only end up becoming a future issue.

EDIT: I'm seemingly not able to get the symbol visible with a version script, as a) it seemingly only operates on non-hidden symbols, so it seemingly fully ignores main and b) it hides all the other symbols that were previously explicitly marked visible. Any ideas if this can somehow be fixed without requiring the main function to be annotated with attribute(visibility(default))?

[jpakkane]

Assuming we merge this can the listed issues be fixed later once they actually come up without breaking bw compatibility with this change?

[sp1ritCS]

The only real issue that can be fixed is the one about the hidden main symbol (the other ones are more "technical limitations" one has to live with).

The hidden main symbol should be fixed however (as the problem will be encountered immediately due to the gtk4-demo) because adding android_exe_type to meson - that just makes built applications useless because they don't provide any symbols doesn't - make much sense.

I've wondered if it may be possible to generate and add with -include a header that declares the main() function with visibility default. The issue with that is, that is somehow has to cope with main(void), main(int argc, char **argv) and main(int argc, char **argv, char **envp).

[jpakkane]

Can the change be done basically with this:

#ifdef __ANDROID__
__attribute__((visibility("default")))
#endif
int main(...) {
    ...

If yes, this seems like the simplest solution.

[sp1ritCS]

Sure that'd work too (realistically the __ANDROID__ isn't even needed), but it would require affected users to update their code.

[jpakkane]

Another thing that came to mind. Would it be possible to implement this in a simpler way? Namely:

• In func_executable call check if we are building for Android and the exe type is set to application
• if yes, remove android_exe_type from the kwarg list and call func_shared_library

This would save us from adding a lot of if/elses in the code as the current implementation does.

[sp1ritCS]

Sure, that might work too. It would however require us to strip any executable specific kwargs (gui_app, win_subsystem, export_dynamic?), not just android_exe_type, as shared_library would complain about it otherwise

I guess I can just hard code them for now, but the current solution with what kwargs apply to specific targets compared to all targets (and build_target) isn't particularly scalable.

[jpakkane]

There should be a named method like strip_exe_specific_kwargs that is called. Then it would be both readable and easily fixable in the future if things change.

[sp1ritCS]

I'm facing a weird issue with that, and I suspect its due to annotation magic which I don't understand.

I have:

def _strip_exe_specific_kwargs(self, kwargs: kwtypes.Executable) -> kwtypes._BuildTarget:
    kwargs = kwargs.copy()
    for exe_kwarg in _EXCLUSIVE_EXECUTABLE_KWS:
        del kwargs[exe_kwarg.name]
    return kwargs

@permittedKwargs(build.known_exe_kwargs)
@typed_pos_args('executable', str, varargs=SOURCES_VARARGS)
@typed_kwargs('executable', *EXECUTABLE_KWS, allow_unknown=True)
def func_executable(self, node: mparser.BaseNode,
                    args: T.Tuple[str, SourcesVarargsType],
                    kwargs: kwtypes.Executable) -> build.Executable:
    for_machine = kwargs['native']
    m = self.environment.machines[for_machine]
    if m.is_android() and kwargs.get('android_exe_type') == 'application':
        return self.func_shared_lib(node, args, self._strip_exe_specific_kwargs(kwargs))
    return self.build_target(node, args, kwargs, build.Executable)
    
[...]
    
@permittedKwargs(build.known_shlib_kwargs)
@typed_pos_args('shared_library', str, varargs=SOURCES_VARARGS)
@typed_kwargs('shared_library', *SHARED_LIB_KWS, allow_unknown=True)
def func_shared_lib(self, node: mparser.BaseNode,
                    args: T.Tuple[str, SourcesVarargsType],
                    kwargs: kwtypes.SharedLibrary) -> build.SharedLibrary:
    holder = self.build_target(node, args, kwargs, build.SharedLibrary)
    holder.shared_library_only = True
    return holder

but running the testcase results in the following:

[...]
  File "/home/florian/workspace/Android/meson/mesonbuild/interpreterbase/interpreterbase.py", line 536, in function_call
    res = func(node, func_args, kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/florian/workspace/Android/meson/mesonbuild/interpreterbase/decorators.py", line 103, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/florian/workspace/Android/meson/mesonbuild/interpreterbase/decorators.py", line 237, in wrapper
    return f(*nargs, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/florian/workspace/Android/meson/mesonbuild/interpreterbase/decorators.py", line 556, in wrapper
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/florian/workspace/Android/meson/mesonbuild/interpreter/interpreter.py", line 1835, in func_executable
    return self.func_shared_lib(node, args, self._strip_exe_specific_kwargs(kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/florian/workspace/Android/meson/mesonbuild/interpreterbase/decorators.py", line 103, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/florian/workspace/Android/meson/mesonbuild/interpreterbase/decorators.py", line 178, in wrapper
    assert isinstance(args, list), args
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError: ('application', ['exe_type.c'])

any idea why?

[bruchar1]

any idea why?

You cannot call func_shared_lib. It is an interpreter function. You should call build_target instead.

[dcbaker]

that assert needs to be None aware t is not None or t in {...}

And please do use the set, the python byte compiler can do some optimization around that which makes the lookup faster.

[dcbaker]

Thanks for making those changes, with the CI turned green this looks good to me.

[sp1ritCS]

@dcbaker I've seen that. I'm just wondering why its necessary, given that interpreter.py should set android_exe_wrapper if it isn't set.

[jpakkane]

   File "/__w/meson/meson/mesonbuild/build.py", line 1999, in __init__
    assert kwargs.get('android_exe_type') in ['application', 'executable']
AssertionError

If the key is not in dict, .get returns None.

[jpakkane]

In case this won't be ready for rc1, it can be merged after that.

[sp1ritCS]

In case this won't be ready for rc1, it can be merged after that.

whats the release schedule like? Any chance I can get #13854 looked at before release? I could work around it (by running meson subprojects update before invoking meson) but it be nicer if I didn't and could save myself from downloading subprojects that I don't need to to my configure options / platform.

[eli-schwartz]

In case this won't be ready for rc1, it can be merged after that.

whats the release schedule like? Any chance I can get #13854 looked at before release? I could work around it (by running meson subprojects update before invoking meson) but it be nicer if I didn't and could save myself from downloading subprojects that I don't need to to my configure options / platform.

Release freeze starts when rc1 is tagged and published. Roughly ~weekly release candidates until we are happy with the results and ready for final release. Only bugfixes and documentation changes allowed for the most part, new features need an explicit Release Freeze exemption from @jpakkane.

That PR you linked may be safe to backport though, even if it misses the final release.