Saving audio bytes transmitted over websocket connection in flask application

Problem background: Develop an application that can open websocket for audio streaming and make connection to client to output speech to text conversion when audio bytes are transmitted over websocket.

Have a python flask backend and javascript frontend to develop an independent websocket application which an provide detection.

Front end code:: index.html

<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.0.1/socket.io.js" integrity="sha512-q/dWJ3kcmjBLU4Qc47E4A9kTB4m3wuTY7vkFJDTZKjTs8jhyGQnaUrxa0Ytd0ssMZhbNua9hE+E7Qv1j+DyZwA==" crossorigin="anonymous"></script>
<script type="text/javascript" charset="utf-8">
    var socket = io.connect('http://localhost:5000');
    socket.on('connect', function() {
        //socket.emit('my event', {data: 'I\'m connected!'});
    });

</script>
<div><h1>Welcome to microphone streaming test</h1></div>

<video id="video" autoplay></video>

<script async src='../static/js/capture.js'></script>

Above is html file which get user devices mic to get audio using navigator

Capure.js

const video = document.getElementById('video');
debugger
function funSendsocketData(buffer){
    debugger;
    socket.emit('my event', {data: buffer});
}
function startup(){
navigator.mediaDevices.getUserMedia({
    audio:true,
    video:false
}).then(stream=>{
video.srcObject = stream;
recorder = new MediaRecorder(stream);  
if(recorder.state === "recording"){
    recorder.stop();
}
else{
    recorder.start(5000);
    recorder.addEventListener('dataavailable', (async event => {

        if (typeof event.data === 'undefined') return;
        if (event.data.size === 0) return;   
        debugger;

        event.data.arrayBuffer().then(buffer =>
            funSendsocketData(buffer)
        );  
    }));
}



}).catch(console.error)
}
window.addEventListener('load',startup,false)

Captures the data for 5000ms or 5 sec and send blobover websocket connection to be saved or processed.

Flask backend

@socketio.on('my event')
def handle_message(data):
    #blob = requests.data
    import base64
    wav_file = open("temp.webm", "wb")
    decode_string = base64.b64decode(bytes(data['data']))
    with open("audioToSave.mp3", "wb") as fh:
        fh.write(decode_string)
    
    
    wav_file.write(decode_string)

Problem lies in backend as I cannot save the blob into the file to be processed by whisper . My blobs are correctly transferred to backend but I am unable to save the file as webm or wav or mp3.

source https://stackoverflow.com/questions/76384978/saving-audio-bytes-transmitted-over-websocket-connection-in-flask-application