problems with different mimetypes in different browsers (v4)

[DevRichter]

Hey,

i have a problem with the different encoding of the files. As far as i can tell the mime-types are different for each scenario: audio/webm (recorded with chrome, also different depending on which browser you record with.) for browser and audio/aac (with different sigantures) from android or apple devices.

We record the voice messages in different devices or different browsers and need to support as many platforms as possible, we upload them via aws s3 as the pure base64 string, then download them as these strings, create an instance of an Audio HTML object and play them. (current state)

The problem is that these files have for example different durations then the real duration or do not get played at all (only in safari an issue). The encoding of voice messages recorded by the android devices seem to work in every browser (we tested) for some reason.

Do you know of a way to record the voice messages with an encoding in e.g. Chrome that will work in Safari? Is there a way to convert this encoding, or record with a more broad possibilty to choose the encoding we want?

[tchvu3]

I've added a part in the readme that discuss these type of issues exactly.
unfortunately, there is no one type that is supported by all major browsers
and phones (at least not yet).
that's why the plugin returns different mime types for each platform.

for now, the best way to deal with this issue is to convert each file to your required format
before you upload it to s3 (you'll need another js library for that),
as it seems that keeping the base64 as-is will not be good enough.
besides, I think it makes sense to add the ability to choose
the mime type when you start recording, I will definitely add it as a parameter soon.

also, I will check again if there is a mime type that is supported by all
major browsers, the chance is slim but maybe over the last year things have changed
since the last time I've checked.

[nkalupahana]

@tchvu3 What about audio/mpeg (mp3)? It looks like it's supported in all major browsers, if I'm understanding these docs correctly, but it's not included as a possible web mime type.

update: never mind -- if you run MediaRecorder.isTypeSupported("audio/mpeg") or anything similar on Chrome, it always returns false. Seems like it only supports webm, which is crazy, but oh well.

[DevRichter]

As we are using this plugin in production, we have now implemented a solution where we transform all audio files via ffmpeg to mp3 then save them as base64 strings and then all of our supported platforms can play them. I think @nkalupahana suggestion is the correct approach, if a viable implementation can be found for it.

[Eraldo]

Seems related: #34 :)

[shyamal890]

@DevRichter once we get the base64 data how are you saving it as a file. I tried to save base64 data with FileSystem API but it seems once its saved with following code, I am not able to transform the file format with ffmpeg.

Here's the code:

let result = await VoiceRecorder.stopRecording();
let blob = base64StringToBlob(result.value.recordDataBase64, result.value.mimeType);
let data_text = await blob.text();
let extension = mime.getExtension(result.value.mimeType);
let write_result = await Filesystem.writeFile({
            path: `temp.${extension}`,
            data: data_text,
            directory: Directory.External,
            encoding: Encoding.UTF8,
        });
        let uri = write_result.uri;

ffmpeg.exec(`-i ${uri} -vn -c:a copy temp.mp3`,(response)=>{
    console.log(response);
},(error)=>{
    console.log(error);
});

ffmpeg just errors out. Most probably because the file is not saved in a format that ffmpeg can process any suggestions would be appreciated.

[DevRichter]

@shyamal890 sorry for the super late answer. for anyone thats still wondering, we are using @ffmpeg/ffmpeg

and then just

const convertAudioToMp3 = async (voiceMessageFile: Uint8Array): Promise<Uint8Array | undefined> => {
    const ffmpeg = createFFmpeg({
        log: true,
    });
    const uuidOriginal = `${randomUUID()}${FILE_SUFFIX}`;
    const uuidConverted = `${randomUUID()}${FILE_SUFFIX}`;
    try {
        await ffmpeg.load();
        ffmpeg.FS('writeFile', uuidOriginal, voiceMessageFile);
        await ffmpeg.run('-i', uuidOriginal, uuidConverted);

        return ffmpeg.FS('readFile', uuidConverted);
    } catch (e) {
        console.error(errors.ERROR_WHILE_CONVERTING_TO_MP3, e);
    }
};