Release v0.4.3

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.35230, 2022.33910 and 2023.11880. It runs on both Mac + PC.

Ensure you select "Enable External .tox" when dragging the MediaPipe component into a new project, else your toe file size will be massive and saves will be very slow. See the TouchDesigner docs for more info:

A word about external files in 2023.10k builds

🎉 Official Full Feature Release! 🎉

⚠️Breaking change & Improvement: Hand tracking

• Left_hand_velocity and Right_hand_velocity in the helpers CHOP have been renamed to h1:hand_velocity and h2:hand_velocity respectively. This is now consistent with the rest of the stats, as the h1 and h2 flags are just "first detected hand" and "second detected hand" and there is no guarantee that hand h1 or h2 is left or right. (i.e. you could have 2 left hands, 2 right hands, or h1 could be right, and h2 could be left)
• Added h*:Leftness and h*1:Rightness scores for each detected hand in the helpers output CHOP - this provides a confidence score for whether each hand is left or right allowing you to actually check for left or right hands! Note this will be the wrong way around if you have "Horizontal flip" enabled for your input (i.e. Left will be detected as Right and vice versa) #86
• Fix Max Number of Hands selector (still only 2 hands max) #70

image_segmentation_tracker & MediaPipe demo toe

• More clearly label the SpoutCam input stages so it's (hopefully!) easier to understand
• Clean up keying in multi-class mode
• Add a segmentation_demo.toe

Objects

• Object detection was always loading the Lite model. Now loads the correct one selected, meaning you should get better results with selecting heavier models. It also makes it (a bit) easier to use custom model files. See #81 for more.

Misc

• Disable audio output from webrender TOP to speed up boot #89

Bumped JS dependencies:

• mediapipe 0.10.14 #92 (no real new features)
• yarn 4.4.0 (see Upgrading from Yarn 1.x.x step-by-step for how to upgrade)
• vite 5.4.1

Release v0.4.2

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.35230, 2022.33910 and 2023.11340. It runs on both Mac + PC.

🎉 Official Full Feature Release! 🎉

Image Segmentation

• Fix segmentation sizing issue

Release v0.4.1

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.35230, 2022.33910 and 2023.11340. It runs on both Mac + PC.

🎉 Official Full Feature Release! 🎉

MediaPipe component

• Improvements to the webcam startup process to make it faster and more reliable to fix #60 and #42
• Hopefully a fix for some webcams starting in 4:3 with a black bar down the side

pose_tracking

• Pose resorts back to original format. Multi pose appends new channels
• Fixed pose multi channel output
• 1 euro filter added to poses to smooth them out a lot, only adding a tiny amount of delay
• Enabled 2 pose tracking. It's very jittery and not true multi-pose tracking, but it does more than it used to! See #62 for more info.
• Lots of general tidy up in the internal network

New feature - image_embeddings

Lets you create a numeric representation of an image in real-time, which is useful in accomplishing various ML-based image tasks.

face_mapping_example

Uses a constant material now, which blends the camera feed w/ the original video much smoother.

Bumped JS dependencies:

• mediapipe 0.10.9
• yarn 1.22.21

Release v0.4.0

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.35230 and 2022.33910 and runs on Mac + PC.

🎉 Official Full Feature Release! 🎉

⚠️ Breaking change to pose tracking ⚠️

pose_tracking now exports 2 poses with p0: and p1: prefixes. In order to re-use your existing pose tracking OPs, add a Select CHOP to select p0:* poses. e.g. left_elbow:y is now p0:left_elbow:y channel. This may well expand to more than just p0 and p1 in the future, so please plan your channel selects accordingly!

MediaPipe component

• Improvements to the webcam startup process to make it faster and more reliable to fix #60 and #42
• Hopefully a fix for some webcams starting in 4:3 with a black bar down the side

pose_tracking

• 1 euro filter added to poses to smooth them out a lot, only adding a tiny amount of delay
• Enabled 2 pose tracking. It's very jittery and not true multi-pose tracking, but it does more than it used to! See #62 for more info.
• Lots of general tidy up in the internal network

New feature - image_embeddings

Lets you create a numeric representation of an image in real-time, which is useful in accomplishing various ML-based image tasks.

face_mapping_example

Uses a constant material now, which blends the camera feed w/ the original video much smoother.

Bumped JS dependencies:

• mediapipe 0.10.9
• yarn 1.22.21

Release v0.3.3

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.35230 and 2022.33910 and runs on Mac + PC.

🎉 Official Full Feature Release! 🎉

MediaPipe component

• Option for auto-assigned network port now the default - port is assigned by OS to avoid conflicts (enabled by default)
• Websocket path parameter removed to tidy up UI
• Webcam start error now works more reliably #42
• Webcam error asks if you have the webcam running somewhere else (most common cause of webcam failing to start in MP)
• Hopefully less eternal loading screens!
• Error message clears if webcam successfully starts
• Error message if local files are missing and you downloaded the dev version

Bumped JS dependencies:

• mediapipe 0.10.8
• vite 4.5.0
• typescript 5.2
• yarn 1.22.20

face_tracking

• Supports up to 5 faces at once 🎉
• Blend shapes and transformation matrix now output properly for facial landmarks #54

Pose

• Supports multi-pose (needs example parsing multiple poses)

object_tracking

• Object tracking bounding boxes fixed #36
• Max number of objects to detect now works correctly

face_detector

• Face tracking bounding boxes and confidence labels added

General

• Added annotation for Video Device In / Syphon Spout Out TOP chain

Release v0.3.2

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.35230 and 2022.33910 and runs on Mac + PC.

🎉 Official Full Feature Release! 🎉

New features

• Smooth Pose Tracking with MediaPipe update to 0.10.7. no more jittery poses
• Camera flip toggle on MediaPipe component flips the camera + output
• Face mesh example using the source video as texture
• Easily track pinch gestures with the new hand_tracking pinch midpoint, position, rotation and distance

Updates

• Include detailed error message for webcams larger than 720p
• Update to MediaPipe 0.10.7 - now with smoother pose tracking!
• Websocket and web server components combined to use a single network port
• Results-processing components (e.g. hand_tracking, face_detector, etc.) are now embedded in main toe file so changes will be saved with the main project - no external tox saves required any more #46
• MediaPipe tox file is still external as it's so big and will make save files huge if embedded.

Bug fixes

• Component reloads every few seconds if the webcam fails to start to hopefully address some instances of "forever" loading screens #46
• Confidence values now work properly
• Defaults to port 3002 to (hopefully) avoid conflicts #29

Release v0.3.1

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.35230 and 2022.33910 and runs on Mac + PC.

🎉 Official Full Feature Release! 🎉

New features

• Smooth Pose Tracking with MediaPipe update to 0.10.7. no more jittery poses
• Camera flip toggle on MediaPipe component flips the camera + output
• Face mesh example using the source video as texture
• Easily track pinch gestures with the new hand_tracking pinch midpoint, position, rotation and distance

Updates

• Update to MediaPipe 0.10.7 - now with smoother pose tracking!
• Websocket and web server components combined to use a single network port
• Results-processing components (e.g. hand_tracking, face_detector, etc.) are now embedded in main toe file so changes will be saved with the main project - no external tox saves required any more #46
• MediaPipe tox file is still external as it's so big and will make save files huge if embedded.

Bug fixes

• Component reloads every few seconds if the webcam fails to start to hopefully address some instances of "forever" loading screens #46
• Confidence values now work properly
• Defaults to port 3002 to (hopefully) avoid conflicts #29

Release v0.2.2

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.35230 and 2022.33910 and runs on Mac + PC.

🎉 Official Full Feature Release! 🎉

Fixes

• updated server memType, which should fix infinite loading issue on some windows computers

Release v0.2.1

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.33910 and runs on Mac + PC.

🎉 Official Full Feature Release! 🎉

Updates

• Object transformation in the facial landmarker example.
• Object tracking outputs info dat
• Object tracking displays object text with percent score in example

Fixes

• Huge performance boost on image classification switching to python instead of json DAT
• I love you gesture now works for hand tracking

Release v0.2.0

⚠️ Make sure to download release.zip to run the TD plugin⚠️

The source code (zip) contains the development environment to create the plugin.
This has been tested on TD 2022.33910 and runs on Mac + PC.

🎉 Official Full Feature Release! 🎉
This version includes an updated image segmentation that utilizes GPU acceleration.