Send Audio Data for Live Streaming Audio Transcription but no response.

[hieuminh65]

System

• Expo 49 SDK
• Latest Deepgram SDK

Bug

• I try to send raw audio data to Deepgram for audio transcription base on the docs Live Streaming Audio Transcription but it does not return me any transcript result or any error. The connection is opened. The data is sent.
• I wonder if Deepgram process base-64, but if not there should be error return.
• Because Expo does not support sending raw audio data through WebSocket so I have to use a library but it only support sending base-64 audio encoded.

Code

LiveAudioStream.on('data', (data) => {
        if (connection && connection.getReadyState() === 1) {
          console.log("Connection established, starting recording..");
          var chunk = Buffer.from(data, 'base64');
          connection.send(chunk);
        }
      });

• No error and result is returned.

[lukeocodes]

Our API does not support base64, sorry.

You might try decoding it before sending. Something like this may work? (Untested code)

const bytes = atob(base64);
const byteArrays = [];

for (let i = 0; i < bytes.length; i++) {
    byteArrays.push(bytes.charCodeAt(i));
}

const byteArray = new Uint8Array(byteArrays);

[hieuminh65]

Hi @lukeocodes,

I was able to send raw audio but still no response. I wonder if you know why:

const startConnection = async () => {
    const deepgram = createClient(process.env.EXPO_PUBLIC_DEEPGRAM_API_KEY);

    const connection = deepgram.listen.live({ model: "nova-2-conversationalai" });

    connection.on(LiveTranscriptionEvents.Open, async () => {
      connection.getReadyState() ? console.log("Connection opened") : console.error("Connection failed to open");
      setButtonRecording("Start");
      LiveAudioStream.on('data', (data) => {
        if (connection && connection.getReadyState() === 1) {
         // convert back to raw audio
          var chunk = Buffer.from(data, 'base64');
          connection.send(chunk);
        }
      });

      await startRecording();
    });

    connection.on(LiveTranscriptionEvents.Close, (event) => {
      console.log("Connection closed", event);
    });

    connection.on(LiveTranscriptionEvents.Transcript, (results) => {
      console.log("Received transcription results", results);
    });

    connection.on(LiveTranscriptionEvents.Metadata, (metadata) => {
      console.log("Received metadata", metadata);
    });

    connection.on(LiveTranscriptionEvents.Error, (error) => {
      console.error("An error occurred", error);
    });

    connection.on(LiveTranscriptionEvents.Warning, (warning) => {
      console.warn("Received a warning", warning);
    });

    return connection;
  };

[lukeocodes]

Can you confirm the data in the buffer is indeed the audio type you're expecting?