TensorFlow with Google Coral.ai

[grasponcrypto]

Wondering if there is a way to utilize the coral.ai Edge TPU for facial/object recognition.

[alextran1502]

I am working on both features for local hosted services. Once that done, I will be working on an option for external services to be used for those machine learning services since a lot of people has their own model trained already 😃 I will keep your suggestion in mind

[Schluggi]

Any updates? :)

[mertalev]

The model engine we use, ONNX Runtime, doesn't currently support Coral (relevant issue). I'm happy to add it if/when this becomes an option.

[afrojewelz]

looking forward to see that outcome

[ptichalouf]

I'm here if U need some tester :)

[KillahB33]

Excited to see the transition to python for the ai piece which hopefully means that this will be coming soon enough!

[pdarcos]

I guess there is still no news on this from upstream?

[bo0tzz]

You would need to check the linked upstream issue for that.

[nikefate]

Feature Request for ONNX Runtime is closed.  Is there any possibility of obtaining support for Coral in another way?

[bo0tzz]

I think you're looking at the wrong upstream issue, it's still open.

[bo0tzz]

We investigated this again recently and the Coral is just not a capable enough device to run the models we use for Immich, so I'm closing this feature request.

[ncridlig]

Just followed this thread after acquiring a Coral from a buddy. Thank you for investigating the possibility!

[Timi7007]

ONNX seems to support Hailo modules, like the new Raspberry AI module, though. Would this be an option?

[mertalev]

I recently got a Hailo module for testing, but haven't started implementing support for it yet. I'll have more time to work on this in a few weeks.

[d9media]

eks.

Awesome! Is there anything in this regard you could use help with? I'm reasonable familiar with docker and microservices but not really an expert in ML. If there's anything i could contribute I'd be glad to. I feel like like this setup could have some potential.

[mertalev]

I think the work will mainly be in exporting the models to Hailo format and making the machine learning service able to use them. I've been meaning to make the model exporting process more automated, so that's something I could use help with. There are several scripts for this that I have to run in the right order, some for each model, and with the Hailo models in the loop it'll get more tedious.

A tricky thing with hardware acceleration is also that it sometimes works on one machine but not on others. After I get things working locally, it'd be great if I could get a sanity check that it actually works.

[d9media]

@mertalev Is this a good pick going forward you reckon? https://buyzero.de/products/raspberry-pi-ai-hat-13-26-tops?variant=45993772089611 Apparently it's their successor of the original AI Kit and now based on Hailo 8L, and the larger on based on Hailo 8.

[mertalev]

Oh nice, I didn't know they came out with new versions of the kit. The 26 TOPS version seems especially interesting.