Add custom interpolation support per sample for scalar tracks

[nfrechette]

At a bare minimum, we need to support constant scalar samples that do not interpolate with their neighbours. Something like a stepped curve where the value changes in discrete steps in some parts but not all of the curve.

We need to store this extra bit within the data format to be efficient. In the track metadata, we can store 1 bit (free) to denote whether it contains any constant keys, and if it does, each key will have an extra bit of packed data. This way, tracks without constant keys do not need to store the overhead AND when decompressing a single keyframe, every bit we'll read will be contiguous.

This extra bit per key can later be extended to a custom header per key to support bezier (or other cubic) interpolation.

How to support this on the raw data side? We need to specify per track/sample which ones are constant. Perhaps we need a new track type for curves where each sample has extra information. Alternatively, a track could have optional metadata per sample that lives alongside it. This later approach would allow mixed usage in a track array since curves that have constant samples and those that do not could live together with a single type. This is probably a better approach as with a separate type, we would be forced to compress them into a separate blob leading to the common scenario where we'll have those with and without. In practice, at decompression we do not care how things interpolate internally as long as we get the correct result out. Interpolation then becomes a property of the track that is preserved through the compression process.

That would later allow for us to automatically sample bezier curves to convert them into linear interpolation as needed during compression. However, care must be taken with bezier interpolation because the speed of ACL comes from the fact that samples are uniform. If the raw data contains beziers, samples will not be uniform. However, we can still support constant samples as long as they align with our sample rate/keyframes.

Progress:

• [x] Add extended sample support to raw tracks
• [x] Add sjson clip writer support for extended samples
• [x] Add sjson clip reader support for extended samples
• [x] Add unit test for extended samples
• [x] Add compression support for extended samples
• [x] Add decompression support for extended samples
• [ ] Add conversion support for extended samples
• [ ] Add regression test for extended samples
• [ ] Add pre-process support for extended samples
• [ ] Update documentation

[nfrechette]

Some notes on splines. Great source here: https://www.youtube.com/watch?v=jvPPXbo87ds

Basic continuity:

• C0 continuity means that the spline positions are not disjoint
• C1 continuity means that the spline positions and velocity are not disjoint
• C2 continuity means that the spline pos/vel/acceleration are not disjoint
• C3 continuity means that the spline pos/vel/accel/jolt are not disjoint Geometry continuity:
• G0 is equivalent to C0, positional continuity
• G1 continuity means that the spline tangents are not disjoint (normalized velocity)
• G2 continuity means that the spline curvature (comb) is not disjoint
• G3 continuity means that the spline torsion is not disjoint For the splines we care about, Basic continuity Cn implies Gn (no 0 speed/tangent flip). We have 3 ways to arrange our control points since a bezier fully passes through P0 and P3 We can have broken tangents where P1 and P2 form a sharp angle with their corresponding tangents from the next bezier (of the spline they form).
• This is C0 continuous but not C1.
• This is G1 continuous We can have aligned tangents where they form a line but of unequal segments.
• This is C0 continuous but not C1. We can have mirrored tangents which form a line with equal length segments.
• This is C0/C1 continuous. This means that when we are mirrored, P4 (P1 form next bezier) can be reconstructed from P2 and P3 by reflecting it (P4 = 2*P3 - P2)

A linear spline has 2 control points and we form a straight line in between using linear interpolation.

• Only ever C0 continuous
• Tangents are implicit
• We pass through all control points: P0 and P1 A Cubic Bezier spline has 4 control points. The curve passes through P0 and P3.
• C0/C1 continuous
• Tangents are manually specified through the control points P1 and P2.
• We pass through some control points: P0 and P3 A Cubic Hermite splint has 2 control points and their desired tangents. The curve passes through P0 and P1 and it has P0' velocity at P0 and idem for P1.
• C0/C1 continuous by design since we supply the desired tangents
• Hermite to Bezier is trivial:
  • B0 = P0
  • B1 = P0 + P0' / 3
  • B2 = P1 - P1' / 3
  • B3 = P1
• Tangents are explicit through P0' and P1'
• We pass through all control points: P0 and P1 A Cubic Cardinal spline has 2 control points and the desired velocity at P0. The curves passes through P0 and P1.
• The desired velocity at P0 is computed automatically using the neighbors and scaled.
• Because we use the neighbors to compute the velocity, we have no value for the start/end of the spline and must add ghost start/end points to extend it.
• A special variant of a Hermite spline A Cubic Catmull-Rom spline is a special case of a Cardinal spline where the scale factor is 1/2.
• C1 continuous
• Tangents are implicit using the neighboring points
• We pass through all control points but we need 2 additional points at the boundaries A Cubic B-Spline has 4 control points but the curve does not pass through any of them
• C2 continuous
• Tangents are implicit using the neighboring points
• We pass through none of the control points
• Important for things that are acceleration sensitive, such as an animated camera

Splines are not curves. From a set of control points, using a spline function, you generate a curve. The same control points using different spline functions yield different curves. Because these splines are all cubic, for a given curve, it is possible to represent it using any cubic spline function with its own set of control points (e.g. curve fitting).

A uniform spline spends an equal amount of time in each spline segment while a non-uniform spline has a varying amount of time spent per segment (e.g. keyframe reduction).

In conclusion:

• For raw data storage, we care about the spline and its control points, we might even need non-uniform support for ultimate flexibility, Hermite is popular for this (e.g. Unity I think)
• For compressed storage, uniform linear spline is fine if our sampling rate is reasonable for the data we care about
• Constant samples cannot use any of the splines above because they are not C0 continuous
• For decompression purposes, we could store everything as Bezier or Hermite as they both allow for sharp corners (e.g. C0 but not C1) and either can convert into the other. The other splines can also be converted as needed. For raw data, we could do the same at the cost of numerical stability (e.g. no longer having the original source curve values if they are Catmull-Rom)
• We could store our data pre-multiplied with the characteristic matrix, it could speed up decompression
• For scalars and scales/translations, curves are somewhat trivial, but what about rotations? For rotations, we can't treat each component individually as they are quaternions. So we need a cubic version?