fortISSimO icon indicating copy to clipboard operation
fortISSimO copied to clipboard

Split "cell catalog" between "main grid" and subpatterns

Open ISSOtm opened this issue 7 months ago • 2 comments

[!NOTE] Help is wanted because I do not have the motivation to implement this; I am however willing to provide guidance to anyone having that motivation. Feel free to contact me.

Context / Motivation

Currently, the biggest problem with using fO over hD is the dreaded Error: The song has NNN unique cells, the max is 256! error.

In the interest of size, fO collects cells (note+instr+FX) into a "catalog", and stores cells as indices into this catalog; in the interest of size and speed (again), those indices are single bytes, which inherently forbid more than 256 entries in the "catalog".

Implementation

A way to relax the limitation is to split the catalog in two[^split]: one for the "main grid", and one for subpatterns. Then the limitation only applies to each one individually, which should be productive considering that typical cells for patterns and subpatterns are quite different.

[^split]: Other options have been considered for splitting, such as one catalog per channel; but these are very impractical to implement, at least with fO in its current form.

As far as fO itself is concerned, this is really simple: duplicate the pointer to .cellCatalog, and change the variable being loaded from in TickSubpattern.

Issues

Song export

First off, teNOR currently assumes that it can compute the "overlap" optimisation by comparing the cells themselves; this wouldn't become true anymore, because nothing guarantees a given cell would have the same ID across both catalogs.

Therefore, cells should instead be compared via some kind of IDs, of three kinds: shared, main only, and subpattern only. Then, the shared cells would get the same indices across both catalogs (so the first few, for simplicity).

Output size

There is also a size problem: there is already somewhat considerable padding within a cell catalog[^padding], and splitting the catalog in two like that would multiply the padding by 2.5!

This could be mitigated by fusing the two catalogs like they currently are if that's possible; then both labels are identical, and only a single header byte is wasted—but that increases the implementation's complexity significantly.

[^padding]: The size of the catalog is 512 + nb_cells bytes, which can be a lot compared to the theoretical minimum size of 3 × nb_cells.

ISSOtm avatar Nov 26 '23 16:11 ISSOtm