charm-relation-interfaces icon indicating copy to clipboard operation
charm-relation-interfaces copied to clipboard

Create an eponymous pypi package with pydantic models

Open sed-i opened this issue 2 years ago • 4 comments

Proposal

I would like to be able to pip install pydantic models

pip install charm-relation-interfaces
# or
pip install charm-relation-interfaces[alertmanager_dispatch]

so they could be easily imported from a centralized location

from charm_relation_interfaces import AlertmanagerDispatchProviderV1
# or
from charm_relation_interfaces.alertmanager_dispatch.v1 import AlertmanagerDispatchProvider

instead of storing them inside the relation's *.py file.

Background/assumptions

  1. JSON schemas are great for general interoperability, but they are not immediately usable from within charm code. What really makes a difference are pydantic models, and how properties are "static".
  2. The "currency" of charm relation interfaces should be pydantic models. Dataclasses usability is limited to very simple schemas and JSON schemas do not convert into a python object we can work with at dev time.
  3. Charm libs are, and will remain, limited to one file; yet if we choose to split provider/requirer into two separate libs, we may need to repeat the pydantic model in both requirer and provider files.
  4. Creating a dedicated charm lib to hold a pydantic model (or two models) would result in inter-lib dependencies, which is not ergonomic.
  5. Creating a dedicated charm lib to hold pydantic models per charming team (e.g. observability_libs.schemas) would artificially subject an essential part of an operator to the workflow of a particular team, and may be perceived as not inviting by community. It may also give a wrong impression, as interfaces are intended to be unique across the ecosystem.

sed-i avatar Sep 28 '23 20:09 sed-i

I'm going back and forth in my head between this solution and one where we offer a CLI tool to 'fetch' a model on demand and store it on disk.

charm-relation-interfaces fetch-model traefik-k8s ingress --role requirer --version 1 --output ./lib/charms/traefik-k8s/ingress/v1/requirer_model.py

pros:

  • lighter on the pack side
  • easier to maintain

cons:

  • more confusing versioning story: can we be notified when there is an upstream change to the model? But given backwards-compatibility assumptions, how bad is it if we're not?

PietroPasotti avatar Oct 02 '23 13:10 PietroPasotti

offer a CLI tool to 'fetch' a model

Seems excessive to expect a dev to charmcraft fetch-lib and another-tool fetch-model. Charm libs could use PYDEPS to specify how to obtain a model from a pypi package.

sed-i avatar Oct 02 '23 17:10 sed-i

Surprisingly, the following works:

$ pip install git+https://github.com/canonical/charm-relation-interfaces/
$ python3
>>> from interfaces.ingress.v1.schema import ProviderSchema
>>> 

Curious how pip decided to install it under interfaces, and why we can't do the same for the "tests" folder:

>>> from tests.test_unit import test_build
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'tests'
>>> 

In any case, this means that we can pip install and import schemas now :)

cc: @simskij

sed-i avatar Dec 14 '23 16:12 sed-i

interfaces is a pure namespace package so it gets installed that way we don't need to publish it on pypi for it to be installable, but it would give it a more production-ready vibe :)

PietroPasotti avatar Jan 02 '24 08:01 PietroPasotti