hypothesis icon indicating copy to clipboard operation
hypothesis copied to clipboard

Improve support for new and user-defined Numpy dtypes (e.g. `np.dtypes.StringDType`)

Open Zac-HD opened this issue 1 year ago • 2 comments

(previously: https://github.com/HypothesisWorks/hypothesis/issues/3950 and https://github.com/HypothesisWorks/hypothesis/pull/3955 got existing functionality working on Numpy 2.0; now we want to support a new feature)

import numpy
from hypothesis import given
from hypothesis.extra.numpy import from_dtype

@given(from_dtype(numpy.dtypes.StringDType()))
def test(x):
    pass

currently raises hypothesis.errors.InvalidArgument: No strategy inference for StringDType(), which is not ideal for something that ships upstream. We could fix that with a special case, but I'd like to support user-defined extension dtypes too if possible, which I expect will take some kind of registry system. (I spoke to @ngoldbaum at Scipy and confirmed that we can't introspect dtypes from Python, though something Arrow-based might eventually be possible)

We should also improve the error message if you forget to construct an instance: it's all to easy to do and instead you see InvalidArgument: Expected dtype but got dtype=<class 'numpy.dtypes.StringDType'> (type=_DTypeMeta).

Zac-HD avatar Jul 11 '24 01:07 Zac-HD

https://github.com/HypothesisWorks/hypothesis/pull/4226 has a good start on this, which anyone looking to contribute to this issue is welcome to use or take over. (There's nothing fundamentally wrong with the linked PR, I'm just unlikely to see it through to completion in the near future).

Liam-DeVoe avatar Jun 03 '25 05:06 Liam-DeVoe

I don't remember the context for the conversation about dtype introspection. Can you share what you'd need from NumPy to get something working without needing to manually register dtypes?

Ping @seberg in case you're interested.

ngoldbaum avatar Jun 03 '25 13:06 ngoldbaum