Skip to content

Conversation

@nasif-co
Copy link
Contributor

Implements the depth-anything-v2-small model in ml5 using transformers.js. This current draft does the following:

  • Maintains compatibility with the existing ml5.depthEstimation user-facing code and examples.
  • Keeps the logic in a separate class from the tensorflow model.
  • For the time being, imports transformers.js through a script that is appended to the html. Moving forward, when transformers.js gets bundled (in Add Transformer.js as a backend for image classification #289 or another pull request specific to doing just the bundling), this will be replaced.

With this in mind, the pending tasks are:

Load transformers by injecting a script to the html head.

Implement transformers into the estimateStart mode. Single frame estimation not yet supported. Also, some refactoring to keep code organized is still needed
Expanded the functionality to encompass single image depth estimation as well. This would all benefit from a refactoring, to keep code as clean and reused as possible, without repetition.
Reorganizing to have two separate classes
Organizing to keep both depth estimation classes separate
Reintroduces depth estimation using transformers but now refactored as a separate class. Still missing some features related to parameters that are currently accepted, like dilation factor
Removed remnants from the tensorflow class, added a utils folder with methods and variables used by both classes, and tested the module\'s features
@nasif-co nasif-co requested a review from AidanNelson January 18, 2026 03:11
@nasif-co nasif-co self-assigned this Jan 18, 2026
@nasif-co
Copy link
Contributor Author

For ease of use, here are two examples on the p5 web editor using a build of this code:

You can swap back to the tensorflow depth estimation on these examples by replacing this:

depthEstimator = await ml5.depthEstimation('depth-anything-v2-small');

with this:

depthEstimator = await ml5.depthEstimation();

This build lacks automatic detection of the best device and dtype, so it is only guaranteed to run on Chromium-based browsers. This is the next issue to be addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants