Handy Work

A module for doing efficient real-time pose detection from WebXR Hand Tracking using Web Workers to ensure it doesn't interfere with the main thread

The pose tracking module is Framework Agnostic, it doesn't rely on any particular library it should work just as well with THREE as BabylonJS or Play Canvas.

Poses can be found in the /poses/ folder and additional poses are welcome

There is also an AFrame module which handles pose tracking and displaying hand models.

Hand Pose Module Usage Example

import {
  update as handyWorkUpdate,
  loadPose
} from "../build/esm/handy-work.js";

loadPose('relax', '../poses/relax.handpose');
loadPose('fist', '../poses/fist.handpose');
loadPose('flat', '../poses/flat.handpose');
loadPose('point', '../poses/point.handpose');

// In RAF
handyWorkUpdate([controller1, controller2], referenceSpace, frame, callback);

Using via a CDN

If you use it via a CDN because it uses a Worker you need to use the standalone version which has the Worker encoded as a string.

import('https://cdn.jsdelivr.net/npm/[email protected]/build/esm/handy-work.standalone.js')
.then(function ({
  update,
  loadPose,
  dumpHands
}) {
  const handyWorkUpdate = update;
  const dumpHands = dumpHands;
  const loadPose = loadPose;

  loadPose('relax', POSE_FOLDER + 'relax.handpose');
  loadPose('fist', POSE_FOLDER + 'fist.handpose');
  loadPose('flat', POSE_FOLDER + 'flat.handpose');
  loadPose('point', POSE_FOLDER + 'point.handpose');
  loadPose('horns', POSE_FOLDER + 'horns.handpose');
  loadPose('shaka', POSE_FOLDER + 'shaka.handpose');
  loadPose('vulcan', POSE_FOLDER + 'vulcan.handpose');
}.bind(this));

handy-work

Documentation for using build/esm/handy-work.js or build/esm/handy-work.standalone.js

• handy-work
  • .loadPose(name, url) ⇒ Promise.<void>
  • .getPose(name) ⇒ Promise.<Float32Array>
  • .setPose(name, pose) ⇒ Promise.<void>
  • .resetHands()
  • .dumpHands()
  • .generatePose(inputSources, referenceSpace, frame) ⇒ void | Float32Array
  • .update(inputSources, referenceSpace, frame, callback)

handy-work.loadPose(name, url) ⇒ Promise.<void>

Loads a pose from the Web the recommended way of using this API

Kind: static method of handy-work
Returns: Promise.<void> - Resolves once it has been completed

Param	Type	Description
name	string	Name of the pose
url	string	URL of the pose to download and add to the registry

handy-work.getPose(name) ⇒ Promise.<Float32Array>

Get a pose from the registry

Kind: static method of handy-work
Returns: Promise.<Float32Array> - A copy of the pose from the registry in the worker

Param	Type	Description
name	string	of the pose to fetch

handy-work.setPose(name, pose) ⇒ Promise.<void>

Get a pose from the registry

Kind: static method of handy-work
Returns: Promise.<void> - Resolves once the pose has been uploaded to the worker

Param	Type	Description
name	string	of the pose to set
pose	Float32Array	Array buffer with the information about the current pose

handy-work.resetHands()

Reset the current tracking performed automatically when the device returns from sleep

Kind: static method of handy-work

handy-work.dumpHands()

On the next frame save the current hand pose to a file

Kind: static method of handy-work

handy-work.generatePose(inputSources, referenceSpace, frame) ⇒ void | Float32Array

Get a pose from the current hand position

Kind: static method of handy-work
Returns: void | Float32Array - The generated pose buffer for the pose.

Param	Type	Description
inputSources	XRInputSource	Array of inputs you want to generate inputs for, requires a left AND right hand
referenceSpace	XRReferenceSpace	Current reference space
frame	XRFrame	Current active frame

handy-work.update(inputSources, referenceSpace, frame, callback)

Kind: static method of handy-work

Param	Type	Description
inputSources	Array.<XRInputSource>	The inputs you want to do pose tracking for
referenceSpace	XRReferenceSpace	The reference space for your scene
frame	XRFrame	The current active frame
callback	function	This gets called with an Array of Arrays with the poses and their distances