Original compression algorithm leaving parts uncompressed?

[grenoult]

Hi there,

Thanks for writing the specifications. I'm playing with a personal project of mine: it's a Sim City 2000 file editor (demo: https://sc2k-save-editor.pages.dev/). My goal is to be able to open, edit and save any sc2 file but I'm struggling at the saving part. It seems the compression algorithm doesn't compress some parts. Details are below:

Summary

• Something missing in the compression algorithm.
• Tested on FLARANCE.sc2 MISC chunk, appearing in June property tax

Details

I’m trying to uncompress and compress data for FLARANCE city, especially the MISC chunk.

I wrote a unit test to compress the uncompressed data and assert the result should be identical, but it’s failing.

• Tests results details

  ➜  sc2k-save-editor git:(master) ✗ npm test -- RleCompression.test.js -t
  
  > [email protected] test
  > jest RleCompression.test.js -t
  
   FAIL  tests/functional/RleCompression.test.js
    ● Test compression › test compression on MISC chunk (FLARANGE) 213
  
      expect(received).toEqual(expected) // deep equality
  
      - Expected  - 13
      + Received  +  9
  
      @@ -1363,13 +1363,13 @@
          1,
          235,
          208,
          132,
          0,
      -   3,
      +   1,
          1,
      -   234,
      +   129,
          234,
          132,
          0,
          3,
          1,
      @@ -3169,15 +3169,15 @@
          0,
          1,
          35,
          130,
          0,
      -   5,
      +   1,
          4,
      -   255,
      +   130,
          255,
      -   255,
      +   1,
          211,
          129,
          0,
          2,
          2,
      @@ -3248,28 +3248,24 @@
          2,
          250,
          10,
          130,
          0,
      -   5,
      +   1,
          1,
      -   255,
      -   255,
      -   255,
      +   131,
          255,
          129,
          0,
          2,
          3,
          140,
          130,
          0,
      -   5,
      +   1,
          1,
      -   255,
      -   255,
      -   255,
      +   131,
          255,
          138,
          0,
          1,
          1,
  
        25 |     it('test compression on MISC chunk (FLARANGE) 213', () => {
        26 |         const result = rleCompression.compress(uncompressed);
      > 27 |         expect(result).toEqual(compressed);
           |                        ^
        28 |     });
        29 | });
        30 |
  
        at Object.toEqual (tests/functional/RleCompression.test.js:27:24)
  
   PASS  tests/unit/RleCompression.test.js
  
  Test Suites: 1 failed, 1 passed, 2 total
  Tests:       1 failed, 15 passed, 16 total
  Snapshots:   0 total
  Time:        0.829 s, estimated 1 s
  Ran all test suites matching /RleCompression.test.js/i.
  

I only investigated the first case and I could find the following:

1. when compressing [1, 234 ,234]
2. the result is [1, 1, 129, 134]
3. but the expected result (in the original saved file) is [3, 1, 234, 234].

Which doesn’t match the specifications (and the consequent algorithm):

1. [1, 234 ,234] is a series of 2 tuples:
   1. [1] which would be compressed as [1, 1] (1 + n byte version)
   2. [234, 234] which would be compressed as [129, 234] (2 byte version)
2. however is it processed as one block only, instead of 2 tuples.
3. it seems like parts of the data was simply left uncompressed which puzzles me.

Any help would be appreciated, thanks!