Controller Navigation Support

[Saroufim]

It would be great if the library can be navigated with a controller. Currently, it doesn't react to controller input.

[Arcitec]

We have been researching this and it turned out to be much more complex than originally thought. It's therefore put on a extremely low priority. If someone with expertise about this topic wants to help out, please do!

Here are the issues we have discovered:

• The GUI library, GTK, does not listen to gamepad inputs. To solve that, we have to use a separate gamepad input library for Python instead, and manually hook the events into the GUI, implement our own "selection cursor", etc, and make sure it all works properly in the GUI.
• There are over 100 gamepad input libraries for Python, and it would be considerable work to find the most robust, best library. There's libraries based on PyGame, and SDL (the cross-platform input library). Both of them require that we create a SDL "game window" before we are able to read inputs via them, which may be a big issue. There's also libraries like inputs which seems ideal because it doesn't require any external dependencies and doesn't need us to open any "fake game window" to register inputs.
• Sandboxing issues: Gamepads are treated as raw device nodes in /dev on Linux. We therefore need device=all permission which is flagged as a "dangerous permission". There's attempts to make an "inputfd" protocol for Wayland (not merged yet) and a Flatpak portal for gamepad access without such broad permission privileges, but neither of those things are created yet. Linux is years behind Windows when it comes to gamepad input.
• Device access permissions: Linux treats all devices in /dev, even gamepads, as root-access devices. You cannot access gamepads unless you (the user) run the app as root. Apps like Steam work around this via the steam-devices package which you install natively on the device, which adds some host rules to allow unprivileged users to access the gamepads. Neither the native nor the Flatpak versions of Steam work without those rules. But since Cartridges is a Flatpak app, we would not be able to install such rules ourselves.
• Exclusive device access: Gamepads on Linux are not multiplexed. Only 1 app can read them at a time. So if we open the gamepad device for reading, then games won't be able to use the gamepad. We'd have to implement a complex system of listening/unregistering the gamepad constantly based on when the Cartridges app has focus. This also fails if a game is running in the background and currently has gamepad access, which would prevent Cartridges from registering the gamepad listener. Likewise, if Cartridges doesn't unregister its own gamepad listener fast enough when we launch a game, then the game itself may fail to open the gamepad device and won't have any inputs. So by registering a gamepad in Cartridges, we can cause many issues because Linux is so bad at gamepad handling.

These are the rules that Steam has to install to get access to gamepads:

/usr/lib/udev/rules.d/51-these-are-not-joysticks-rm.rules
/usr/lib/udev/rules.d/60-steam-input.rules
/usr/lib/udev/rules.d/60-steam-vr.rules

Those in turn are complex rules which define which devices are joysticks and which ones aren't. We would NOT be able to do something similar in Cartridges. If we implement gamepad access, it would be the user's own job to configure their host to give all non-root users access to read gamepads, and that's way out of the scope of what we'd be able to assist anyone with (just look at the rules in the files above, if you have Steam installed, they're very complex).

So if anyone's got more experience with solving Linux's gamepad situation and the issues above, then please help out! If not, we are unlikely to see gamepad support in Cartridges.

[Mars7x]

Highscore has support for gamepad navigation. I'm not sure about sandboxing, permissions, and exclusive device access, but maybe it could help a little to check out what they did.

[sonnyp]

https://gnome.pages.gitlab.gnome.org/libmanette/

[kra-mo]

Yeah, I've been looking into that.

[kra-mo]

@sonnyp do you by any chance happen to know how you can get libmanette working inside of Flatpak? I haven't had much luck.

Edit: Nevermind, as most things in life, it was a problem with my build configuration :)
Its working now.

[sonnyp]

@kra-mo in case it's helpful: https://gitlab.gnome.org/GNOME/gnome-build-meta/-/merge_requests/2217

(you can't use libmanette from org.gnome.Sdk)

libmanette also has a working manifest example

And if you need some kind of demo, @halfmexican built one for Workbench here

https://github.com/sonnyp/Workbench/pull/309/files#diff-1b34825a0da7ebbdc205aae6048fba59c443d2959620a958b516f3d72d413930

[PixsaOJ]

If you are using Steam, you can enable Desktop Layout in controller settings.

[kra-mo]

Ah! That's neat. It is actually using libei too, which makes sense.