Waveform and audio not synchronised in label studio 1.7.2

[CrisRodriguez]

Describe the bug In a speech to text annotation task, when listening to the audio it is unsync with the waveform. It happens with :

• <Audio name="audio" value="$audio"/>
• <AudioPlus name="audio" value="$audio"/>

Additional info:

• Audios are stereo.
• It is more evident in long audios (20-30 minutes)
• It does not happen in Label studio 1.4.x

Screenshots

First image in 1.4.1 shows the sync between box and waveform. Second image in 1.7.2 shows the unsync between box and waveform, when listening, the audio is synced with the box, therefore it is the waveform that is unsynced.

Environment (please complete the following information):

• OS: Linux
• Label Studio Version 1.7.2

Thanks,

[bmartel]

Hi @CrisRodriguez 👋. A fix has already been made for this and will be coming in LabelStudio 1.7.3. For the time being, if you are able, you can workaround this and use the latest source version of LabelStudio. The 1.7.3 official release should be out within 1-2 weeks time.

[bmartel]

I will keep this ticket open for visibility, and close upon the official release of 1.7.3.

[CrisRodriguez]

Hi @bmartel,

Thanks for your message.

I got it. Meanwhile I am using the 1.4.x version that works pretty well for my less than 20-minute audios.

Thanks,

Cristian

Cristian

[CrisRodriguez]

Hello @makseq @bmartel,

Can u please confirm that this issue has been solved in 1.7.3, and if yes, close the issue ?

Thanks,

Cristian

[bmartel]

Hey @CrisRodriguez , I apologize for the late reply. So this is fixed in the 1.7.3 release, I just tested the build right now. However, I noticed something incorrect with the feature flag states for this build.

Currently during local testing of 1.7.3 release build, it appears to default to the old audio player (which would also work in your case, just with less performance and improvements). If you want the current (and fixed) audio player in the 1.7.3 release, you will have to set the following flag as an environment variable when running your LabelStudio instance.

ff_front_dev_2715_audio_3_280722_short=1

As well, the config would have to specify the following:

<Audio name="audio" value="$audio" decoder="webaudio" />

As it seems this builds feature flag state has decoder="ffmpeg" as the default option (the option which is producing the issue for you here).

Reference documentation: https://labelstud.io/tags/audio.html#Parameters

Again I apologize for this inconvenience, if this works for you I will close this issue.

[mogwai]

I'm also noticing this on browser and edge on version 1.12:

{
  "release": "1.12.0",
  "label-studio-os-package": {
    "version": "1.12.0",
    "short_version": "1.12",
    "latest_version_from_pypi": "1.12.0.post0",
    "latest_version_upload_time": "2024-05-06T19:31:45",
    "current_version_is_outdated": true
  },

  "label-studio-os-backend": {
    "message": "fix: LEAP-947: Check task when we get a response from ML (#5742)",
    "commit": "b2fb5819f214d06f9300bd6c3a7853c78bdd176d",
    "date": "2024/04/17 15:26:37",
    "branch": "",
    "version": "1.12.0+0.gb2fb581"
  },

  "label-studio-converter": {
    "version": "0.0.58"
  }
}

I've marked here the delay of when I hear the sond and where it is marked as starting. This is using the default html5 audio player and I've tried both decoding techniques.

The delay seems to get worse as the audio is played.

[csantosbh]

I can also reproduce this on label-studio 1.18 (both Audio and AudioPlus), especially with longer audios of 30mins+ (though the sync loss is already noticeable very early on, being at around 3secs one minute into my audio). I've tried both decoder options (webaudio and ffmpeg), but both exhibit the same issue. As a sanity check, I downgraded to 1.4.1 (where I couldn't get proper audio widgets) and 1.7.3, where I was still reproducing this issue.

Below is an example: The short silence close to 00:01:06.250 was audible at around 00:01:03.750:

Sadly this is quite serious and renders labelstudio labels completely unusable in my application. Any idea what could be behind this or other workarounds for 1.18?

Update - workaround found

When using both decoder="webaudio" and player="webaudio", the audio now plays correctly without delay 🚀