mltframework
Audio Spectrum and Audio Waveform displays nothing with Jack
Kubuntu 20.10, Shotcut 20.11.28
Audio Spectrum and Audio Waveform displays are not working when using the Jack audio backend.
I confirm this report is correct. the audio visualization filters require that that the audio is rendered before the video. In my testing, the audiowaveform filter reports:
[filter audiowaveform] Audio not preprocessed.
which means get_image() is being called before get_audio() when jack is enabled.
That happens because JACK audio is integrated as a MLT filter attached to a muted SDL2 audio consumer rather than using the JACK MLT consumer. That is done because JACK transport control is only available in the filter and not the consumer. To understand why JACK is primarily a filter in MLT requires one to appreciate the flexibility of JACK as more than a audio playback API. It might be possible to port all the JACK transport stuff from the filter to the consumer and then have Shotcut switch consumers. However, I was thinking about dropping the SDL2 consumer and replacing it with a new generic one that only exposes callbacks to use Qt audio APIs. We could still do both, but we are talking about a large amount of work for something very niche as it concerns the special combination of JACK, audio visualization filters, and preview in the app. Export is not affected. It is very low priority
I agree this is low priority.
Can you think of a simple way to synchronize the Jack filter so that it pulls the audio before the consumer pulls the video? If nothing simple comes to mind, then I agree this might just wait until we change to a different consumer in Shotcut.