MQA_identifier
MQA_identifier copied to clipboard
purpl3F0x
Reverse engineering the MQA decoder
Topic for Reverse engineering the MQA decoder and maybe a (virtual) renderer.
Από ότι ξέρω, Roon comes with a built-in MQA decoder.
If Roon is written in .NET (I think, it might be), it'd launch the RoonInstaller64.exe via Sandboxie to grab all the files from it—or I would even try just unzipping it with a tool like 7Zip.
I would then use a tool like ILSpy to decompile the .NET libraries into C# and look around...
Yes roon is in .NET. I have decompiled it with dotPeek and didn't found something. Although my C# knowledge stops when the # starts so there might be more there. I would guess maybe in some of the Roon.Audio.***.dll.
There is also Audirvana - it's a UWP application.
At least the way of identifying an MQA from an MQA blueStudio, should be open. YMMV 🤝
I don't find a good reason why they would :P
- Have you read the various Archimago's thoughts on MQA?
- Are you aware of Amazon's choice to not enroll MQA on Amazon Ultra HD?
Even giants like Microsoft started embracing open-source one way or the other; being completely closed-source is a dead-end as history shown us... Still, YMMV.
To be clear, I'm a huge fan of MQA.— (Τελεία και παύλα.)
But I believe MQA (the company) should release the base codec as open-source, while keep the MQA Studio™ one internal to be used only in authorized mastering studios.
They way they set it up right now, makes it impossible for an open-source community of developers and enthusiasts to evolve...
@moodmosaic Please stay on topic. The issue here is to reverse engineer the mqa soft decoder, not a forum to discuss what MQA has to do.
from: https://patentimages.storage.googleapis.com/70/18/e6/c2b80f8d65bebf/WO2013186561A2.pdf
FIG 7.B seems to be MQA decoders path
The noise shaped joiner is described at fig 8.B and the lossless bandjoin is fig 4.B
Lossy Decompression with Touch up to lossless (22) seems like a black box (or not if I stop reading only the pics).
It seems, MQA format is very similar it's predecessor MLP (Meridian Lossless Packing) and Dolby TRUE-HD. So libffmpeg possibly has done some work for us.
For example, sample rate identification, seems to be very similar indeed. MLP:
static inline int mlp_samplerate(int in) {
if (in == 0xF)
return 0;
return (in & 8 ? 44100 : 48000) << (in & 7) ;
}
MQA (pour-me implementation):
uint32_t OriginalSampleRateDecoder(unsigned c) {
const uint32_t base = (c & 1u) ? 48000 : 44100;
const uint32_t multiplier = 1u << (((c >> 3u) & 1u) | (((c >> 2u) & 1u) << 1u) | (((c >> 1u) & 1u) << 2u));
return base * multiplier;
So libffmpeg possibly has done some work for us.
Could that be the reason why there are articles around like this one? (Although this one uses sox instead.)
I don't think so. Probably the mqa dithering helps the resampling a bit. From what I read on the above paper there is no upsampling at all, the lower bits produce a higher frequency content of (17.5bits) and then they are interleaved together.
But he is right Auralic probably does. Probably chi-fi marketing
Sync header for MLP: XORing gives you the header (like MQA 😏) https://wiki.multimedia.cx/index.php/Meridian_Lossless_Packing#Major_sync_header
Probably have to rethink MQA sync word size. I don't like the 36bits
EDIT: MQA like TRUE-HD doesn't have 2 substreams - so safe to assume those blocks are gone (#sad #prayforsubstream1)
I just realised the tidal app ships with the debug .pbd file, lol thank you electorn.
I'm trying to find if there there is a given length for each subframe of the stream. Does seem to. BUT there seem to be a pattern(kinda) on the differences (ignoring the difference of the first one with the second one). One is big (57K-60K samples) and a small one (5.8-6K samples) 🤷♂️
Track 1 (2L-048_14_stereo.mqa.flac)
Original FS: 96000
Samples where MQA subframe starts: [ 4 15905 73232 79335 137827 143779 203927 209378 269233 275177
334561 340281 399310 405649 464483 471042 529811 535667 595132 601211
660267 666444 725456 731087 791141 797541 857111 863396 922459 928899]
Differences
---- 15901
---- 57327
---- 6103
---- 58492
---- 5952
---- 60148
---- 5451
---- 59855
---- 5944
---- 59384
---- 5720
---- 59029
---- 6339
---- 58834
---- 6559
---- 58769
---- 5856
---- 59465
---- 6079
---- 59056
---- 6177
---- 59012
---- 5631
---- 60054
---- 6400
---- 59570
---- 6285
---- 59063
---- 6440
Track 2 (2L-125_04_stereo.mqa.flac)
Original FS: 352800
[ 4 15878 72298 78183 138765 144072 203166 208797 269184 274366
334482 340218 399953 406001 465215 470758 529955 536196 595164 601340
660397 665867 726461 731934 791803 797558 857169 862727]
---- 15874
---- 56420
---- 5885
---- 60582
---- 5307
---- 59094
---- 5631
---- 60387
---- 5182
---- 60116
---- 5736
---- 59735
---- 6048
---- 59214
---- 5543
---- 59197
---- 6241
---- 58968
---- 6176
---- 59057
---- 5470
---- 60594
---- 5473
---- 59869
---- 5755
---- 59611
---- 5558
you guys read these before ? https://audiophilestyle.com/forums/topic/30572-mqa-technical-analysis/#comments
https://code.videolan.org/mansr/mqa/-/tree/master
Seems there are some stuff there 😉. From a quick look, it links the blueOS llinux shared library tho, but it's a start
you guys read these before ? https://audiophilestyle.com/forums/topic/30572-mqa-technical-analysis/#comments
https://code.videolan.org/mansr/mqa/-/tree/master
After cloning, you need to acquire the libbluos_ssc.so file on your own, IIRC.
After cloning, you need to acquire the
libbluos_ssc.sofile on your own, IIRC.
Yep, https://audiophilestyle.com/forums/topic/30572-mqa-technical-analysis/page/21/?tab=comments#comment-695892
Yeah using shared libraries doesn't seem very legal 😬. And still doesn't reverse engineer the decoder.
I had mostly accuired what the above repo does, so I won't need to troubleshoot them I guess. I just need to write them in c++, 'cause I was using python for bootstrapping.
@ValZapod 🙂. They put it inside a zip file, for safety.
I don't have much time to work on it :(. Although there is some progress :P
So I crosscompiled from aarch64 (arm64) to armel (or is it armhf) mqadec and mqarender, but i do not get it how to make it see libbluos_ssc.so? Apperantly puting it in the same folder with binaries is not enough. also, does this only do two first unfolds? third unfold is still not available?
BTW, crosscompilation needs dpkg --add-architecture armhf; apt-get update; apt-get install libgmp-dev:armhf libsndfile1-dev:armhf
And the script that gets library from web.archive needs p7zip-full.
Crosscompilation is done by make CC=arm-linux-gnueabihf-gcc-7 CROSS_COMPILE=arm-linux-gnueabihf- ARCH=arm
btw: Should this ever be finished, or someone know some other opensource decoder for MQA, than it would be nice if note could be given to https://trac.ffmpeg.org/ticket/6158 .
btw: Should this ever be finished, or someone know some other opensource decoder for MQA, than it would be nice if note could be given to https://trac.ffmpeg.org/ticket/6158 .
I told you there that this library for arm 32 bit (libbluos_ssc.so) is quite hard to use. I mean even I cannot! And I do have android tablet with Ubuntu (Galaxy Tab S4) using arm64.
Here is .so and crosscompiled binaries. https://0x0.st/oZl7.zip oZl7.zip
I think it still did not get the third unfold, since the author even says it does not exist and thus three unfold files only decode to 176.4 kHz.
Okay i managed to make it work. I just needed LD_LIBRARY_PATH pointing to a new .so library (in archive above) and then just install libasound2-dev:armhf
It works now. BUT ONLY FIRST UNFOLD.
Okay, for second unfold you need other libraries:
They open all these command line players (like flac) to decode to standardout. They developed their own alsa program, that reads the raw data from the raw output of the flac and madplay command line players, and then pushes it to the alsa layer via their own closed source alsa player:
/usr/bin/dspout
readelf -a /usr/bin/dspout
Dynamic section at offset 0xbca4 contains 28 entries: Tag Type Name/Value 0x00000001 (NEEDED) Shared library: [libbluos_ssc.so] 0x00000001 (NEEDED) Shared library: [libasound.so.2]
They have added their own mqa render to alsa:
/etc/asound/asound.conf.simple
pcm.mqa_render { @args [RATE] @args.RATE {type integer} type rate slave { pcm "digital" rate $RATE } converter ssc }
This ssc converter is loaded via
/usr/lib/alsa-lib/libasound_module_rate_ssc.so
This library contains the proprietary MQA decoder and works like a sample rate converter. It is not illegal to make a derived work of a LGPL based work such as the ALSA framework, but I'm not sure that also means adding a proprietary library to an existing library.
https://0x0.st/oN-m.zip files for above
and this is mqadec'ed file to be mqbscan'ed (not mqascan'ed) and rendered. Third unfold may be possible here on this file after 1st unfold. https://0x0.st/oN-d.xz
from: https://patentimages.storage.googleapis.com/70/18/e6/c2b80f8d65bebf/WO2013186561A2.pdf
FIG 7.B seems to be MQA decoders path
The noise shaped joiner is described at
fig 8.Band the lossless bandjoin isfig 4.BLossy Decompression with Touch up to lossless (22) seems like a black box (or not if I stop reading only the pics).
Can someone explain what "noise shaped joiner" means here? I'd like to help reverse this format but I'm not completely sure how this "folding" works quite yet.
@purpl3F0x Does mqarender works for you? Oogh.
Don't know - just a suggestion
As far as I know 44.1/48K sample rates are unfolded to 88.2 and 96K respectively
OK guys, Newb here so please be cruel......
I haven't tried any of the decoding MQA programs yet but I am looking for a solution to what feels like and upcoming end to MQA altogether.
Recent news has been popping up about MQA going into receivership, with either a buyout or IP being bought and sold into pieces..
I have used TIDAL exclusively for 5 years now and love it. Mainly through my studio monitors and a Focusrite Clarett I feel I am getting better than CD quality, maybe 96Khz. So the software does the first 2-3 unfolds, as far as I am aware. Though when I use my DAC and Beyerdynamic's I am sure it is getting above 96Khz prob minimum 192Khz maybe more if the track allows it.
This must be the FPGA/ARM7 whatever chip utilized, doing some form of Blackbox magic decoding. The mysterious further unfolds.....
So I am trying to find a way to preserve my MQA experience for a long time in the case MQA brand goes bye-bye, Tidal stops supporting it or as many users have found, titles appear then disappear in Master Quality.
So Capture is high on my list of priorities. I have thought about just recording each and every Master Quality song via outputs straight back into inputs of recording gear. Maybe even looping SPDIF for best lossless chance instead of using Mic- Pre's. But this would require live recording- and this will take month's, MQA might be gone the way of the 'Dodo' sooner than later.
Solution 1. Download as many Master versions onto HD's and hope a full unfolding software/hardware player decoder becomes available in the future.
Solution 2. Save everything into FLAC files. ( I am sure there is a difference - something about the time domain, PCM vs DSD and other stuff my brain just doesn't have the capacity to explain at the moment) But this may be the best solution, future proofing.
Solution 3. Open to suggestions from anyone with more experience than myself.
Note: Yes I know I am suggesting Piracy and copyright infringements. But I don't want to lose one of the small pleasures I have.
Firstly, this project is only for educational purposes. Please read GitHub Terms and conditions. Secondly, it's about reverse engineering the mqa decoder/encoder not help pirate the whole sh1t out of TIDAL.