You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Would be interesting to have a generic browser backend. A bit like jupyter_rfb but without Jupyter. It may also make it easier to test / benchmark new techniques.
Can be done by the context, or in canvas._rc_present_xx(), or a bit in both.
Let's assume we have a binary websocket, but if we don't than base64 encoding is part of this step too.
jupyter_rfb does png and jpg encoding.
Making the image size smaller will make the next step faster.
But the encoding also greatly affects the presentation in the browser.
Sending over the network, io:
The speed of this step is pretty good if you're on localhost.
Otherwise, the speed can vary a lot, depending on where the server and browser are.
Speed can vary over time too.
It would be nice to measure this speed and use it to control the encoding steps above.
Presenting in the browser:
This step includes any necessary decoding. Browsers have builtin support for some decoding mechanisms.
jupyter_rfb sets the src of an <img> object to the base64 encoded png/jpg. This is surprisingly fast.
Using a canvas. IIRC drawing an image to a canvas is pretty slow.
Using WebGL / WebGPU.
There are multiple paths to go through these steps. What's tricky is that a choice in one step affects the other steps, so
we cannot simply pick "the best" from each step and stack them together.
MP4 has nice properties in being pretty good in a situations with low/variable bandwidth, and it very naturally improves the resul over time for static images.
Something like WebCRT can possibly be used to implement some of these steps.
The text was updated successfully, but these errors were encountered:
Would be interesting to have a generic browser backend. A bit like
jupyter_rfb
but without Jupyter. It may also make it easier to test / benchmark new techniques.Overview in short
The simplified and naive approach:
More detailed steps and notes on performance
See also pygfx/wgpu-py#378
We can also use a custom present-method, so that the image can be encoded. That broadens the options quite a bot. Assuming rendering with wgpu here:
canvas._rc_present_xx()
, or a bit in both.src
of an<img>
object to the base64 encoded png/jpg. This is surprisingly fast.There are multiple paths to go through these steps. What's tricky is that a choice in one step affects the other steps, so
we cannot simply pick "the best" from each step and stack them together.
MP4 has nice properties in being pretty good in a situations with low/variable bandwidth, and it very naturally improves the resul over time for static images.
Something like WebCRT can possibly be used to implement some of these steps.
The text was updated successfully, but these errors were encountered: