Audio chunk leakage between streams

[phamson02]

I'm working on a server to generate audio using the Coqui engine. I've initialised a single instance of CoquiEngine. For each synthesise request, I create a TextToAudioStream object using this initialised CoquiEngine. I've noticed that with concurrent requests, audio chunks are leaking into the wrong streams. Is this behaviour expected with Coqui?

[SreevaatsavB]

Facing the same issue here, keen to understand about how to handle concurrent requests. In sesne, the process of stopping the on-going generation of audio chunks, terminating it so that no more chunks will be generated and sent to client and then starting a new process for new text input.

[KoljaB]

Oh, my bad for missing this earlier!

This issue relates to (code example)

In short, you can't use the same engine for multiple concurrent requests because they'd all end up writing to the same audio queue. To handle true concurrent synthesis, you’ll need multiple engine instances.

With local engines this means way more VRAM usage. There's no way around that for proper concurrent inference. TTS is super resource-heavy, so you'll need a fast GPU, and the engine setup has to be spot-on.