Using the ONNX runtime

[robclouth]

Have you experimented with the ONNX runtime for inference? In my initial tests it runs RAVE models about twice as fast.

[christhetree]

Hi Rob, sorry for the delay, I've been on vacation the last two weeks. We've been keeping an eye on the ONNX runtime and in theory would love to use it. However, when we first built Neutone almost 2 years ago, there were issues with maintaining state (we use script() instead of trace() when exporting since we need to potentially capture dynamic runtime graphs) and with the STFT in ONNX that made it not usable for our purposes. However, you're right that ONNX should be much faster in many cases, potentially at the cost of flexibility which is a tradeoff we are always juggling. If you have any ideas or experiences with ONNX we would love to hear them!

[robclouth]

No worries! Hope you enjoyed it. I'll keep experimenting with it an let you know. Also it compiles easily as a static lib so no more dlls in system on windows...
Could you answer my issue when you get chance please? I've hit a wall.
#49