xarray icon indicating copy to clipboard operation
xarray copied to clipboard

Typing of abstract base classes

Open headtr1ck opened this issue 3 years ago • 4 comments

This PR adds some typing to several abstract base classes that are used in xarray.

Most of it is working, only one major point I could not figure out:

What is the type of NDArrayMixin.array??? I would appreciate it if someone that has more insight into this would help me.

Several minor open points:

  • What is the return value of ExplicitlyIndexed.__getitem__
  • What is the return value of ExplicitlyIndexed.transpose
  • What is the return value of AbstractArray.data
  • Variable.values seems to be able to return scalar values which is incompatible with the AbstractArray definition.

Overall it seems that typing has helped to find some problems again :)

Mypy should fail for tests, I did not adopt them yet, want to solve the outstanding issues first.

headtr1ck avatar Sep 11 '22 10:09 headtr1ck

The inheritance in the indexes is quite convoluted. Trying to fix the runtime issues concerning abstract methods missing opens another rabbit hole ... Don't know if I can fix that without some type:ignores or Anys :/

headtr1ck avatar Sep 11 '22 18:09 headtr1ck

There's an error around the abstract transpose method, which is causing lots of tests to fail, but that aside let's merge!

I tried removing the abstract transpose method from the Baseclass but then several mypy complaints appeared. I think one has to add overloads to as_indexable, which however I failed to to. I got complaints that np.ndarray and pd.Index have overlapping types, might be a mypy bug?

headtr1ck avatar Sep 12 '22 06:09 headtr1ck

What is the return value of ExplicitlyIndexed.getitem

I think this is the mess I ran in to in a different PR: https://github.com/pydata/xarray/pull/6874#discussion_r946154605 . It's basically another ExplicitlyIndexed array except when it wraps a BackendArray, in which case you get a concrete array. IMO it should always be an ExplicitlyIndexed array.

dcherian avatar Sep 12 '22 15:09 dcherian

I have now removed the requirement to implement a transpose method as only the adapter classes implemented it but not the backend classes (it required two # type:ignores...). Maybe it would make sense to have two separate base classes for these two use cases?

headtr1ck avatar Sep 12 '22 17:09 headtr1ck

I think the implementation of a protocol for duckarrays should be done in another PR.

The mess of ExplicitlyIndexed.__getitem__ should also be fixed in another PR, I don't feel comfortable enough yet with this part of xarray to touch things there :)

headtr1ck avatar Dec 16 '22 21:12 headtr1ck

Already one problem is showing from "enforcing" a shape property by setting is as an abstract property. cfgrib comes with its own backend which is used instead of xarrays internal one and it is still using a shape attribute.

Anyone actually know why a abstract property cannot be fulfilled by setting an attribute? At the end I require at least a readable variable, but do not care whether it is settable or not...

Alternative: Do not use abstract properties but the "old-fashioned" way of raising a NotImplementedError in the ABC instead.

headtr1ck avatar Dec 16 '22 22:12 headtr1ck