[Question/discussion] Can register_entities only be called once during setup?

[tdamsma]

I am trying to grasp how to deal with modularized code that defines entities in different places. As far as I understand from the code one is supposed to call the register_entities function once, from the alembic env.py. Now I have my code split up in several modules, is there a way I can have each module register their own entities? Or should each module expose a list of entities, and then I need to import those in my env.py?

In the spirit of SQLAlchemy it would be nice to register an entity as instantiation and be done with it (for SQLAlchemy that means derive from Base). Perhaps it would be possible to add a similar mechanism that registers an entity on each subclass instantiation, e.g. using the init_subclass mechanism.

[olirice]

is there a way I can have each module register their own entities?

you'll want to keep the call to register_entities in env.py because

• it should only be called once
• due to python's import system, if you accidentally didn't import the location where register_entities was called, none of those entities would be visible to alembic

Or should each module expose a list of entities, and then I need to import those in my env.py?

that would work. it doesn't have any opinion about how you would want to expose the entities to env.py

Perhaps it would be possible to add a similar mechanism that registers an entity on each subclass instantiation, e.g. using the init_subclass mechanism

that would be an option, but requiring a user to collect all entities in env.py explicitly & deciding which ones to add in register_entities, was an intentional design decision. It allows user's to put arbitrary rules in env.py for configuration (e.g. changing the schema name dynamically). Its also the least confusing for newcomers.

---

If your project has no import side-effects, there is helper function available that will traverse your package and collect all instances of an arbitrary class into a list.

# env.py
from alembic_utils.experimental import collect_instance
from alembic_utils.pg_view import PGView
from alembic_utils.replaceable_entity import register_entities

import myapp

instances: List[PGView] = collect_instances(myapp, PGView)

register_entities(*instances)

that's your best bet if you want to avoid plumbing lots of imports

[tdamsma]

Thanks for your reply and for the pointers. Perhaps I am not the average new user but I found this mechanism a bit confusing, and would definitely prefer self registering magic to make it work.

On Wed, 11 Aug 2021, 19:38 Oliver Rice, @.***> wrote:

is there a way I can have each module register their own entities?

you'll want to keep the call to register_entities in env.py because

• it should only be called once
• due to python's import system, if you accidentally didn't import the location where register_entities was called, none of those entities would be visible to alembic

Or should each module expose a list of entities, and then I need to import those in my env.py?

that would work. it doesn't have any opinion about how you would want to expose the entities to env.py

Perhaps it would be possible to add a similar mechanism that registers an entity on each subclass instantiation, e.g. using the init_subclass mechanism

that would be an option, but requiring a user to collect all entities in env.py explicitly & deciding which ones to add in register_entities, was an intentional design decision. It allows user's to put arbitrary rules in env.py for configuration (e.g. changing the schema name dynamically). Its also the least confusing for newcomers.

If your project has no import side-effects, there is helper function available that will traverse your package and collect all instances of an arbitrary class into a list.

env.pyfrom alembic_utils.experimental import collect_instancefrom alembic_utils.pg_view import PGViewfrom alembic_utils.replaceable_entity import register_entities

import myapp instances: List[PGView] = collect_instances(myapp, PGView) register_entities(*instances)

that's your best bet if you want to avoid plumbing lots of imports

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/olirice/alembic_utils/issues/56#issuecomment-897019592, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4BSU4WHR45RVRROOIOABTT4KYQRANCNFSM5B65X6JA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[ddemidov]

Would it be possible to keep track of the instantiated functions/views inside PGView/PGFunction constructors? Something like

_pg_functions = []

class PGFunction:
  def __init__(self, *args, **kwargs):
    ...
    _pg_functions.append(self)

Then I guess at the point of the migration generation register_entities(*_pg_functions, *_pg_views) could be called behind the scenes? Of course, this can also be done on the user side, by extending the PGFunction and PGView classes.

EDIT: If there is an auto_register() function that just calls register_entities(*_pg_functions, *_pg_views) and has to be called explicitly by the user inside env.py, then the users could still omit the call and apply their arbitrary rules for registration.

[olirice]

My concerns with any automatic registration approach are:

• they fail unless the ReplaceableEntity is on an import path in env.py
• less configurable / explicit

So long as you don't have import side effects, the snippet using collect_instances above should be enough to get the behavior you're looking for