[Meshcat] Set object from JavaScript code#22683
Open
siddancha wants to merge 8 commits intoRobotLocomotion:masterfrom
Open
[Meshcat] Set object from JavaScript code#22683siddancha wants to merge 8 commits intoRobotLocomotion:masterfrom
siddancha wants to merge 8 commits intoRobotLocomotion:masterfrom
Conversation
Collaborator
|
I'm going to mark this +(status: do not review) until meshcat-dev/meshcat#188 makes progress first. In general, yes Drake should plumb through the full set of commands that Meshcat.js offers, so the only question here is really should Meshcat.js offer that new command. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR implements:
Meshcat::SetObjectFromThreeJsCode(std::string_view path, std::string_view three_js_lambda)that lets the user invoke Meshcat's
set_object_from_codecommand (meshcat-dev/meshcat#188) by sending a string of native JavaScript/THREE.js code. The code string contains a JavaScript lambda function that creates aTHREE.Object3Dinstance; this object is then set atpath. An example command is:meshcat.SetObjectFromThreeJsCode( "/meshcat/my_object", R"""(() => { const geometry = new THREE.PlaneGeometry(...); const material = new THREE.MeshPhongMaterial(...); return new THREE.Mesh(geometry, material); })""" );Motivation
Currently, the only way to set objects in Meshcat is via
SetObject()calls that support a specific set of object types (likeShape,PointCloudetc.). By letting the user create objects via native JavaScript/THREE.js code, Drake's Meschat API can support all THREE.js object types.Relation to #20941
#20941 proposes something like
void SetObjectRaw(std::string_view path, std::string_view threejs_object_json)i.e. passing aTHREE.Object3D's serialized JSON. In my view the following are the pros/cons of sending THREE.js code instead.Pros
Reflectorobject [1, 2] to create a reflective floor, as shown in this demo. However,Reflectorcannot be serialized because it uses ShaderMaterial [3], which in turn uses dynamic textures that are computed at runtime and cannot be serialized. When I try to serialize by callingReflector.toJSON(), I get the following warning:.toJSON()function to understand what the serialization format looks like.Cons
TypedArrays(e.g.Float32Array). For example, setting aPointCloudusually requires specifying a largepositionsarray. Msgpack can efficiently pack float arrays. On the other hand, embedding the floating values into a string might make the string unnecessarily bulky.In conclusion, this PR might not_resolve #20941, but it can handle most uses cases well. For example, the proposed use case of setting text in Meshcat (#20884) is easily handled by this approach, as demonstrated in the test (see below).
Test example
This PR also adds an example to
//geometry:meshcat_manual_testthat creates a floating text mesh usingMeshcat::SetObjectFromThreeJsCode, as desired by #20884. On runningwe see
Resolves #20884
This change is