LintDB icon indicating copy to clipboard operation
LintDB copied to clipboard

Support Float16

Open mtbarta opened this issue 1 year ago • 1 comments

Since late interaction is already so expensive storage-wise, we can support float16 as a quick win to reduce storage costs.

This would involve adding support for a new datatype. Alternatively, we could consider a quantizer that converts float32 to float16. I'm preferential to a new data type.

mtbarta avatar Aug 16 '24 18:08 mtbarta

Let's break this idea up a bit.

We should support float16 for individual FieldValues. For Tensors, quantization should be preferred in reducing storage size.

Also note, we should support float16 computation as well. We can break that out into a separate ticket.

mtbarta avatar Aug 20 '24 19:08 mtbarta