Poor quality sound when GPU/CPU is stressed

[giuliogatto]

Hello, first of all thank you for your great library, which works really well in general.
I found a problem in Chrome Desktop Version 83.0.4103.106 (Official Build) (64-bit): I am trying to record sounds while my app is using Window.requestAnimationFrame() with a machine learning algorithm running inside each frame. The problem I am facing is that .getMp3() function generates a file with very poor quality, full of cracks sounds and skips.. I tried reducing the bitrate to 64 but the same thing happens. Any suggestion on how this could be fixed?
Thanks

[buzinas]

I'd say that the issue most likely happens on the recording phase, and not on getMp3 which just takes what was recorded and encoded and creates a Blob from the buffer.

When you want better performance while executing stuff on the main thread, you usually use a WebWorker to make sure the processing happens on a separate thread. That said, unfortunately, there is no getUserMedia on workers, so we can't listen to the mic in a separate thread.

Since we encode the audio being received in real time, you could try to send the buffer to a worker for encoding to see if this would help unburdening the main thread while it listens to the mic. But I haven't tried that, and honestly don't even know if LameJS (our encoder) works on a worker.

[a-sklyarov]

Here are a few findings:

Part 1

• I managed to re-create a situation in which skips appear in the recording (in browsers on my laptop) by adding a for loop in the encoding code, which counts to, say, 100 million in order to stress the CPU for every onaudioprocess event. The bigger the for loop, the more pronounced the skips in the recording are.
• I solved this by moving the encoding to a Web Worker. I can confirm that LameJS works fine in a Web Worker. Following this, the presence of a for loop did not affect the quality of the recording, only increased the processing time.

Part 2
After "Part 1", users with iPhone still experienced crackling sounds in their recordings. :(

• I managed to re-create a situation where crackling occurs (in browsers on my laptop) by explicitly setting the buffer size to the smallest allowed value of 256. The default in Chrome is 2048 and in Firefox it's 4096. I also confirmed that Safari on iPhone has 256 as the default.
• This made me think that if I set the buffer size to something bigger on iPhone, I can stop the crackling. I set it to 2048. Unfortunately, that didn't help. Crackling still occurs.

[giuliogatto]

Hey, sorry I didn't get back with this. Moving the processing AFTER the recording has fixed the problem. It's obviously a stressed CPU problem and not much can be done about it.

[a-sklyarov]

Thanks @giuliogatto I should probably open a separate issue about the crackling recording on iPhone because it isn't fixed by moving the processing to a Web Worker or after the recording.

[buzinas]

@giuliogatto Would you mind opening a PR with your fix? I'd be glad to encode the recording after it's done instead of real-time. It looks like a good trade-off to have better audio quality.

[buzinas]

@giuliogatto friendly bump, just in case you can share your code with us.

[giuliogatto]

Hey, sorry @buzinas for the delay: what I meant is that I didn't use this library but DIRECTLY "lame.js" https://github.com/zhuker/lamejs AFTER recording in raw audio format and BEFORE sending data to the server

[buzinas]

Got it. Thanks, @giuliogatto!

[alectrocute]

Users still having issues with this, even after I moved lame encoding to after the recording, FWIW.

[alectrocute]

I think this may be a #13 dupe