zarr.js icon indicating copy to clipboard operation
zarr.js copied to clipboard

feat: add (optional) float16 support

Open manzt opened this issue 3 years ago • 2 comments

Fixes #127

This PR implements optional support for float16 datatypes by inspecting globalThis.Float16Array. Rather than adding https://github.com/petamoriken/float16 as a dependency, users must opt into this behavior by adding Float16Array as a global in their environment.

import { openArray } from 'zarr';

const z = await openArray({ store: "http://example.com/f2data.zarr" });
await z.getRaw(null) // throws: '<f2' is not supported natively in zarr.js. In order to access this dataset you must make Float16Array available as a global. See https://github.com/gzuidhof/zarr.js/issues/127

import { Float16Array } from '@petamoriken/float16';
globalThis.Float16Array = Float16Array;
await z.getRaw(null)  // { data: Float16Array, ... }

manzt avatar Sep 01 '22 18:09 manzt

cc: @keller-mark

manzt avatar Sep 01 '22 18:09 manzt

From the typescript perspective, there is a more “correct” way of communicating this behavior, which I will add to this PR https://github.com/DefinitelyTyped/DefinitelyTyped/blob/fb66211010980641f01bcfe8ffbc0cc3d71989ca/types/ndarray/index.d.ts#L43

manzt avatar Sep 02 '22 15:09 manzt

Ok, I've changed the TYPED_ARRAY_MAPPING to inspect globalThis on import only, and also removed the global declaration from zarr. This way it is up to the end user to add the global declaration and be explicit about the type.

with globalThis.Float16Array defined in user code (TypedArray union includes Float16Array).

image

without globalThis.Float16Array defined in user code (TypedArray union does not include Float16Array).

image

ref: https://github.com/DefinitelyTyped/DefinitelyTyped/pull/56163#discussion_r732279580

I'll merge and add some additional docs.

manzt avatar Sep 06 '22 16:09 manzt