Using a Compute graph as alternative to Observables

[SimonDanisch]

This introcudes ComputePipeline (all names very WIP) as an alternative to Observables, which solves:

• some performance problems
• multi input/output situations
• updating multiple attributes at once
• thread safety and better clean up
• Hopefully better stacktraces with clearer input/output mappings
• Maybe better compile times and inference

State of this PR:

Primitive plots

Implementations for primitive plots tend to go through stages. First is an initial implementation, where simple plots like scatter(rand(10)) work. The second stage includes connecting various attributes that have not yet been connected and fixing errors. The last stage is having all tests pass, which is not in reach until the pr is close to done.

Backend	Initial implementation	most/all attributes attached	Tests pass
GLMakie			text, mesh, scatter, image, heatmap, surface, voxels, meshscatter, volume, lines, linesegments
WGLMakie			text, heatmap, surface, mesh, meshscatter, volume, voxels, scatter, lines, linesegments, image
CairoMakie			text, scatter, lines, linesegments, image, heatmap, surface, mesh, meshscatter, voxels

General

TODO:

• dim_converts
• SpecAPI
• old robj cleanup causes camera caching to desync in GLMakie (can be skipped once everything is implemented)
• how to handle primitives that also act as recipes (text, mesh with MetaMesh)
• how to handle ShaderAbstraction Sampler/Buffer types
• how to handle local updates in voxels (mabe with Sampler?)
• handle light transformation per scene, not per plot
• how to deal with failed updates in renderloop (i.e. avoid erroring every frame)
• texture atlas/distancefield handling (different resolution between GLMakie and WGLMakie, exists as user attribute)
• handling of (cached) camera matrices in CPU projections
• lighting docs
• WGLMakie camera relative lights
• CairoMakie overrides
• don't skip safety checks when attaching camera matrices (no unsafe_register)
• text linesegments for LaTeXStrings
• fix countourf updates (plot mostly disappears)
  • this is caused by Computed-forwarding not triggering onchange updates in the child graph
• reconsider text argument handling/naming (Can we make plot.text and plot.position(s) the inputs regardless of the actual arguments? Rather than sometimes that, sometimes arg1?)
• fix typing issues in recipe graphs
  • e.g. cycler in "Errorbars x y low high" trying to set a color::Computed.value::Symbol = RGBAf(...), barplot lowclip highclip nan_color trying to set highclip to a Symbol
• remove outdated data_limits() and boundingbox() methods
• make sure there are no memory leaks due to connected graphs (i.e. nodes not getting gc'd due to being connected to a parent graph)
• move_to needs to do some surgery to disconnect the plot compute graph from its parent scene compute graph and connect it to another without loosing dependencies
• check if pixel_space is needed in any plot, remove it if not
• check if surface generates incorrect normals with distorting transform funcs (and model?) (we no longer consider transform_func in mesh gen in CairoMakie and WGLMakie)
• empty plotlist errors in backend

Breaking:

• lights no longer have Observable fields. Instead they are updated through interface functions
• attr = copy(Attributes(plot)); pop!(attr, ...); attr[...] = ... style of preparing passthrough attributes does not work anymore
• plot!(parent, args..., parent.attributes...) syntax does not work anymore
• kwargs now takes priority over attr when doing plot!(parent, attr, args...; kwargs...)
• replace_automatic! does not work with ComputeGraph + Observable defaults
• p = text(string); p[1] is no longer a String, but the position
• remvoed annotations!() (which have been deprecated in favor of text!())

Maybe breaking:

• Something like @recipe BarPlot (x, y) used to work with PointBased conversions, but doesn't anymore (removed in barplot, waterfall, tooltip)

State of this PR: first implementation for various plots, hacking into the plot pipeline to replace the their constructors and hijack it with a computegraph implementation for convert_arguments/convert_attributes and any backend computation. It currently leaves the plot type unchanged and stores the compute graph in plot.args[1] (lol) and overloads a few methods already just for Scatter (not getproperty etc though, yet).

Next steps:

• axis dim converts
• get/setproperty
• more benchmarks
• WGLMakie/CairoMakie test
• Same treatment for lineplot, which should really help see any performance gains

Continuation of: #4550 and #4360

Notes

• Consider removing deprecated String-like args in text in compute release or all non-position args in the next breaking release after (i.e. also Vector{Tuple})
• text shouldn't billboard as it applies per glyph, breaking text layouting

[ffreyer]

Progress on GLMakie:

The number of failing tests is now down to 15, with 27 tests throwing errors.

TODOs/Problems:

• Legend & Colorbar integration
• dim_converts integration
• SpecAPI integration
• dynamic px_per_unit changes (e.g. through save) (this may not work with camera matrix caching as it is implemented here)
• scatter may not match billboarding behavior of master, as rotation isa Billboard never happens. Needs tests on master to compare
• axis3d!() sometimes draws wrong, maybe a transformation issue?
• transform overwrite don't work yet

[ffreyer]

CairoMakie is now at 2 fails - 28 errors, GLMakie at 14-27

[ffreyer]

This should be useless because update_limits!() already skips updates when the current scaling is acceptable.

[ffreyer]

Just a thought related to #4752 - would it make sense to have "versioned" nodes in the compute graph?

For example with linesegments we have color -> scaled_color -> synched_color. What if instead they were all just named "color" and internally treated as different nodes acting as versions of the name. When requesting "color" you'd get the latest version of it, so that in the backend the detail of what the Makie does disappears. This might be as simple as making graph.outputs[name] and array where each index is a new version.

This would make the pipeline easier to adjust because naming would become easier. E.g. if I wanted to hack the linesegments pipeline to do something to synched_color before rendering, I would have to rename that node and add a computation creating a new "synched_color". With versions it you could just add another computation from color (V3) to color (V4) and you'd be done.

That would be quite useful for transform functions that change geometry as described in #4752. Maybe it would also be useful in recipes?

[ffreyer]

This only applies to LineSegments. For them it maps a value per point pair to two values for the vertices, which results in per-segment colors/linewidths. In Lines this would result in a const value segment, then a gradient segment, then a const value segment, etc.

[ffreyer]

When switching between GLMakie and WGLMakie this should error because we'll try to re-register the computation with "atlas_1024_32" instead of "atlas_2048_64" or vice versa. If it doesn't this may cause incorrect uvs to be used because they may be sampled from a different atlas.

[MakieBot]

Benchmark Results

SHA: e858b8eca5ac2801b1daa3ff7658dedc5db88fad

Warning

These results are subject to substantial noise because GitHub's CI runs on shared machines that are not ideally suited for benchmarking.

[bjarthur]

just want to say thank you for all the work you put into the new compute graph! my website is amazingly fast now!!