DIRAC icon indicating copy to clipboard operation
DIRAC copied to clipboard

Making use of Python typing

Open chrisburr opened this issue 3 years ago • 3 comments

Now that DIRAC is Python 3 only we can think about making use of Python's typing fuctionality. This was added in Python 3.5 however over the past few years it has grown to be really powerful and is even capable of understanding DIRAC's S_OK/S_ERROR convention (#6309).

A few ideas:

  • [x] Add CI to run mypy (waiting for a release with generic TypedDict support https://github.com/python/mypy/issues/13385)
  • [ ] Add tests to ensure that ReturnValues.py is correctly typed (based on the example in #6309)
  • [ ] Add type annotations to diraccfg. Ideally this should include dropping Python 2 support.
  • [ ] Add type annotations to gConfig
  • [ ] Fully type the contents of the CS so it's possible to statically validate (maybe impractical)
  • [ ] Allow RPC calls to be more richly typed for HTTPS
  • [ ] Try using MonkeyType or pyannotate to add more type hints (maybe wait until we have a little more experience rather than adding wrong hints)
  • [ ] Consider using typeguard or beartype to validate type hints at runtime when running tests

To expand on the last point, we could have a function like:

def export_something(dict[int, int]) -> dict[int, str]:

Currently the keys would become str due to the use of JSON however if the RPC call took advanage of the type hints we could easily cast to the correct type and remove the need to do type validate in every handler method. pydantic would likely be the best way to implement this, similarly to how fastapi works.

chrisburr avatar Aug 15 '22 08:08 chrisburr

If we do down this way, should we consider running https://github.com/Instagram/MonkeyType or https://github.com/dropbox/pyannotate?

fstagni avatar Aug 17 '22 12:08 fstagni

I hadn't considered that would even be a possibility. For bits with non-trivial DIRAC semantics (S_OK/S_ERROR/CFG/executeRPC/dict) we'll need hand crafted types (like https://github.com/DIRACGrid/DIRAC/pull/6309) but for the long tail of "easy" cases that looks like a good idea. Once mypy is updated (eta late August) and we have CI for it I'll take a look.

There is also the option of running tests with runtime type checking via tools like typeguard or beartype which might be interesting.

chrisburr avatar Aug 18 '22 05:08 chrisburr

I'd move this to diracx

fstagni avatar Oct 26 '23 14:10 fstagni