mara
Refactor mara flask CLI commands
The cli commands returned by the function MARA_CLICK_COMMANDS are automatically added to the Mara Flask App (code see here).
There is a default way in Flask to add extra commands from a module by using a entry_point in the module package with name flask.commands: [...]. See here. This functionality is already available since 1.0.0 (pallets/flask#2259)
I suggest to deprecate the current pattern of using the MARA_CLICK_COMMANDS function in advance to the flask standard way of dealing with this.
In addition, I suggest to redesign the command names / and keep the old names only for upgrade compatibility. The command flask mara_pipelines.ui.run could be e.g. changed into <my_mara_app> pipeline run by using a click group.
I did a bit discovering in the code and came on the folllowing idea:
When the MARA_CLICK_COMMANDS function returns a click.Command instance, it should behave as of today: The command is made available as <package_name>.<command_name> e.g. flask mara_pipelines.ui.run.
When the MARA_CLICK_COMMANDS function retunrs a click.MultiCommand instance or a derivate of it, it should pass over the multi command itself. With that it would be possible smoothly to integrate a new logic.
For example:
import click
@click.group
def pipeline():
pass
@pipeline.command
def run(...):
# ...
pass
MARA_CLICK_COMMANDS = [pipeline]
would then result in the following CLI command:
flask pipeline run
Using a custom script named mara would then result in the following call:
mara pipeline run
I had similar ideas a few years ago (https://github.com/jankatins/mara-cli + https://github.com/jankatins/mara-config/) but entry points were quite slow at that point. Not sure if that got better since.
Maybe I don't get your point here about the entry points ... I never experienced performance issues with entrypoints. I see people using it in all sort of projects
About the config (mara-config): I see that that is something which we should tackle. I really like the idea how superset does it: It has a superset_config.py file which looks like a .env, but it allows you additionally to use custom python code in it. The file is imported via a flask app core function which takes care of the rest. I don't see the big benefit with this config patching but maybe there is something I miss here...
Another thing I have in mind is to replace mara-acl with Flask-AppBuilder. It is just too much for the mara project to develop all this logic with Authentication, Authorization, user groups management etc. I would rather use an external tool for this (which btw. adds the cli command group flask fab ... to let you manage user accounts from the command line).