Develop a voice input feature similar to WeChat

[jerryokk]

Clear and concise description of the problem

Create a voice input feature that allows users to record audio by long-pressing a button for up to 60 seconds. Users can cancel the recording by sliding up and releasing the button to send the audio. After sending the audio, the feature will use OpenAI's audio API to convert the audio into text and send it to ChatGPT.

长按录音，最大录音60s，上滑取消，松手发送。发送后通过 openai 的 audio 接口转成文字发送给 chatgpt api。

Suggested solution

const formData = new FormData();
formData.append("file", audioFile);  // WebM 录制的文件
formData.append("model", model);
formData.append("prompt", prompt);   // 提示中写使用简体中文，否则默认出来的是繁体的
formData.append("response_format", "json");
formData.append("temperature", temperature);
formData.append("language", language);

fetch(BASE_URL + "/v1/audio/transcriptions", {
    method: "POST",
    headers: {
        "Authorization": "Bearer " + API_KEY
    },
    body: formData
})
.then(response => {
    if (!response.ok) {
        throw new Error("HTTP error " + response.status);
    }
    return response.json()
})
.then(json => {
    // 转写的文本在 json.text
})
.catch(error => {
    console.error(error);
});

Alternative

No response

Additional context

OpenAI's audio API charges $0.006 per minute for transcription.

Validations

• [X] Follow our Code of Conduct
• [X] Read the Contributing Guide.
• [X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

[jerryokk]

@yzh990918 Expert, evaluate whether this is an interesting suggestion.

[Root-FTW]

Clear and concise description of the problem

Create a voice input feature that allows users to record audio by long-pressing a button for up to 60 seconds. Users can cancel the recording by sliding up and releasing the button to send the audio. After sending the audio, the feature will use OpenAI's audio API to convert the audio into text and send it to ChatGPT.

长按录音，最大录音60s，上滑取消，松手发送。发送后通过 openai 的 audio 接口转成文字发送给 chatgpt api。

Suggested solution

const formData = new FormData();
formData.append("file", audioFile);  // WebM 录制的文件
formData.append("model", model);
formData.append("prompt", prompt);   // 提示中写使用简体中文，否则默认出来的是繁体的
formData.append("response_format", "json");
formData.append("temperature", temperature);
formData.append("language", language);

fetch(BASE_URL + "/v1/audio/transcriptions", {
    method: "POST",
    headers: {
        "Authorization": "Bearer " + API_KEY
    },
    body: formData
})
.then(response => {
    if (!response.ok) {
        throw new Error("HTTP error " + response.status);
    }
    return response.json()
})
.then(json => {
    // 转写的文本在 json.text
})
.catch(error => {
    console.error(error);
});

Alternative

No response

Additional context

OpenAI's audio API charges $0.006 per minute for transcription.

Validations

• [x] Follow our Code of Conduct
• [x] Read the Contributing Guide.
• [x] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

OMG !! this would be great!!

[ddiu8081]

Sounds a bit complicated to achieve, but a good feature to use audio api. In v2 I plan to provide a separate panel for the Whisper API, and may not be used in the input box.

[Chanzhaoyu]

using voice in web pages is restricted by https