Audio to Text is not working locally

[lancemao]

Self Checks

• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] Pleas do not modify this template :) and fill in all the required fields.

Provide a description of the help you need

I added ASR in my app which uses Gemini Pro as LLM, when I tried to start audio recording (already granted permission in my browser), I got exception in app\components\base\voice-input\index.tsx

in console, I got

Looks like js-audio-recorder is outdated and maintenance is ceased.

Dify cloud works fine. So I have no idea what went wrong, please help me

[dosubot[bot]]

Hey @lancemao, great to see you diving into another challenge with Dify! How's everything going besides this hiccup?

Based on the information you've provided, it seems like there are two main issues to address: the outdated js-audio-recorder library and the incorrect error handling in the catch block of the handleStartRecord function.

1. Updating the js-audio-recorder Library: If there's an updated version of the js-audio-recorder library that is compatible with your current setup, you should update the library. This can be done by modifying your package.json file or using your package manager's update command. For example, if you're using npm, you can run npm update js-audio-recorder. If the library is no longer maintained and there's no updated version available, you might need to consider replacing it with a more recent or maintained library. Unfortunately, I couldn't find a specific recommendation for a replacement within the Dify repository.

2. Correcting Error Handling in handleStartRecord: The error handling in the catch block of the handleStartRecord function should be corrected to handle the error object e instead of assigning e = undefined. Here's an example of how you might modify the code:

   try {
     // Your code to start recording
   } catch (error) {
     console.error("Error starting audio recording:", error);
     // Handle specific errors here
     if (error instanceof SpecificError) {
       // Handle a specific error, e.g., permissions issue
     } else {
       // Handle general errors
     }
   }
   

   This will ensure that any errors that occur when starting audio recording are properly logged and handled.

After making these changes, you should test the audio recording feature both on Dify cloud and in your local development environment to ensure that the issue is resolved. If the problem persists, it would be helpful to know the exact error message or code that is being thrown when the exception occurs in the handleStartRecord function. Unfortunately, I couldn't find this information within the Dify repository.

For more information on using LLMs with Dify, you can refer to the LLMs Use FAQ in the Dify documentation.

Sources

• en/user-guide/creating-dify-apps/llms-use-faq.md

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[lancemao]

I guess this is due to React debug mode useEffect will be called twice. So if I add:

useEffect(() => {
    initCanvas()
    handleStartRecord()
    return () => {
      recorder.current.stop()
    }
  }, [])

it works fine locally

[crazywoola]

@zxhlyh Please take a look at this.

[dosubot[bot]]

Hi, @lancemao,

I'm helping the Dify team manage their backlog and am marking this issue as stale. From what I understand, you were experiencing issues with audio to text functionality in your app, specifically with ASR and Gemini Pro. Dosubot provided guidance on updating the js-audio-recorder library and correcting error handling in the handleStartRecord function. You mentioned that adding a specific useEffect in React debug mode resolved the issue locally. Crazywoola also tagged zxhlyh to take a look at the issue.

Could you please confirm if this issue is still relevant to the latest version of the Dify repository? If it is, please let the Dify team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you!