Type annotations for `omero.model`

[multimeric]

Currently omero.models is a magic dynamic module whose contents are only created at runtime, as far as I know. This is annoying for two reasons.

The first is that my type checker says I'm doing something wrong when I access omero.model.DatasetI, and the second is that the type checker doesn't know or autocomplete omero.model.DatasetI.some_field.

There are tools for generating type stubs in these scenarios. For example after installing mypy you can run stubgen -m omero.model --inspect-mode which outputs a massive file, a snippet of which is:

class DatasetI:
    def getName(self, current): ...
    def setName(self, _name, current, wrap: bool = ...): ...

The argument types are missing which is a bit annoying, but this would still be a massive improvement. More info on sub generation here: https://mypy.readthedocs.io/en/stable/stubgen.html.

[will-moore]

Wow, yes 28955 rows is massive!

I've no idea how hard this would be to integrate with our code generation of the omero.model package? Would we want it for all the other packages, omero.gateway etc?

Can you use your own copy of the generated __init__.pyi to fix your type-checker?

I've not yet tried using a type checker when working in Python - maybe I should?

[joshmoore]

I've no idea how hard this would be to integrate with our code generation of the omero.model package?

This would need to go approximately here -- https://github.com/ome/omero-blitz/blob/master/build.gradle#L157 -- to be picked up by downstream users.

[multimeric]

I should mention that I don't know why the omero.model module is dynamically generated, and someone with a better understanding of that could comment on whether my solution makes sense here. Generating stubs only really makes sense if there's an underlying native code or if the whole Ice package is out of the control of the OMERO developers.

I've no idea how hard this would be to integrate with our code generation of the omero.model package?

Oh, is there code generation? Can you elaborate on how that works? Maybe you could just modify this system to create the stubs or even the whole module statically so that we don't need stubs.

Would we want it for all the other packages, omero.gateway etc?

The gateway module has some 11,000 lines in it that I can see in my editor, so it's not generated at runtime. It doesn't need type stubs since type aware tooling can just look at that source code. In contrast, omero.model has only like 3 lines that omit all the actual types users will need, which is why I proposed the stubs.

Can you use your own copy of the generated init.pyi to fix your type-checker?

Yes, I can, but I like to push improvements upstream so that other people don't have to configure everything themselves.

I've not yet tried using a type checker when working in Python - maybe I should?

Try opening an OMERO client project in VS Code. The red lines here are telling me I've made a mistake according to the type system, even though I haven't:

[joshmoore]

I should mention that I don't know why the omero.model module is dynamically generated, and someone with a better understanding of that could comment on whether my solution makes sense here.

This is a result of how the Ice generator creates the module(s), namely by creating individual omero_model_<class>.py files.

Generating stubs only really makes sense if there's an underlying native code or if the whole Ice package is out of the control of the OMERO developers.

The latter. The slice2py generator is owned by Ice and written in C++ (i.e., not highly extensible).

Oh, is there code generation? Can you elaborate on how that works? Maybe you could just modify this system to create the stubs or even the whole module statically so that we don't need stubs.

Yes, lots and lots of code generation. All defined within the omero-blitz repository that I linked to previously.

Yes, I can, but I like to push improvements upstream so that other people don't have to configure everything themselves.

:100:

[multimeric]

Okay so if we can't touch the Ice generator or the files that it generates, then I think my stub approach makes sense?

If the stubs themselves are committed to the repo, then people in the future can tweak the stubs to gradually add more detail, e.g. marking getName() as returning a str.

[joshmoore]

Okay so if we can't touch the Ice generator or the files that it generates, then I think my stub approach makes sense?

Off-hand, I think so! :+1:

[multimeric]

A review of #450 would be great!