Low-latency input events in workers

[marcoscaceres]

Session description

The following proposal is based on this email and related discussion in the Gamepad API repo. Posted here as Chair of the WebApps WG.

This proposal for a W3C breakout session aims to address a critical limitation of the web as a platform for applications requiring low-latency response to user inputs. Despite technological advancements, the web's current input handling mechanisms are inadequate for such applications due to their dependence on the main thread.

User input APIs, which include keyboard, pointer, gamepad, HID, USB, MIDI, Bluetooth, and serial interfaces, are currently designed to register handlers on the main thread. This design results in inconsistent and unreliable response times, as the main thread is frequently occupied with other processes. This limitation adversely affects a broad range of applications, particularly those requiring precise user inputs, such as certain video games, Digital Audio Workstations (DAWs), synthesizers, and art programs that rely on touch gestures.

While incorporating timestamps in input APIs could offer some improvement, a more effective solution is to allow these APIs to be accessible from Web Workers. This change would significantly reduce input latency issues. Some APIs are already making progress in this direction, but others, especially those tied to the DOM API like keyboard and pointer events, face more complex challenges due to their inherent link to the main thread.

This issue, recently discussed with folks involved with the Gamepad API, highlights the need for a comprehensive approach that encompasses various APIs. Therefore, this session proposal seeks to gather community input and collaboration to explore and develop solutions that enhance the web's ability to handle user inputs with the necessary low latency. This effort requires a broad perspective and collective action, addressing the current limitations and identifying feasible paths for improvement in web application responsiveness.

Session goal

See if it's feasible to route some HID events to workers

Session type

Breakout (Default)

Additional session chairs (Optional)

@reillyeon, @nondebug, @7ombie

IRC channel (Optional)

#low-latency

Other sessions where we should avoid scheduling conflicts (Optional)

No response

Instructions for meeting planners (Optional)

No response

Agenda (link or inline) (Optional)

No response

Link to calendar (Optional)

Calendar entry

Meeting materials (Optional)

• Minutes

[mustaqahmed]

We explored some aspects of the problem a few years ago. Dumping the docs below in case they are still helpful:

• Events in Workers and Worklets: lists a few different use cases, and provides a technical deep dive into a proposed event dispatch idea.
• Input Events for Worker Threads: compares a few possible API alternatives, and also gives two polyfill-based demos.