Dash.FileFormats icon indicating copy to clipboard operation
Dash.FileFormats copied to clipboard

Remove dependency to Neptoolia.DataLayer.Decompressor

Open MysteryDash opened this issue 8 years ago • 3 comments

MysteryDash avatar Aug 31 '17 22:08 MysteryDash

I have recently written a C++ based set of tools, including support for (de)compression for the DW_PACK format. Porting the Code to C# should be rather straightforward

Focus Parts:

Decompressing

  1. The Compressed data is divided in Chunks of N (usually 0x40000) bytes
  2. Read a serialized Huffman Tree
  3. Create a Cursor on the three, then read bit by bit and move left/right until you hit a Leaf Node
  4. Append that node value to the decompressed data, move to previous step until no data is left

Compressing

  1. The Decompressed data is divided in Chunks of N (usually 0x40000) bytes
  2. Each chunk is Analyzed to create a Huffman Tree
  3. The tree is serialized in the compressed data
  4. For each byte of the uncompressed data in the chunk, append the bits of the key for that byte (as representated in the Huffman Tree)
  5. Repeat previous step until no data is left

The code i wrote to compress/decompress is parallelizable (due to its chunked nature), and makes use of async/futures in C++.

Similarly it should be able to be ported to Tasks without much issue, or simply made to be executed in a sequential fashion as well.

usagirei avatar Oct 24 '17 17:10 usagirei

Hello @usagirei,
That's awesome, to be honest, this part was the most difficult for me, since Neptoolia.DataLayer.Decompressor is basically a copy-paste of the game's decompression code. And converting C/C++ to C# is what I already did for DXT (de)compression.
I'll take a look and port it as soon as possible. Thank you !

MysteryDash avatar Oct 24 '17 18:10 MysteryDash

No problem, feel free to contact me (or actually reply here and i'll check, since i don't have contact information set on github) if you have any questions

usagirei avatar Oct 24 '17 20:10 usagirei