dbt-fal icon indicating copy to clipboard operation
dbt-fal copied to clipboard

Ad-hoc script running in fal runtime

Open chamini2 opened this issue 2 years ago • 0 comments

Is your feature request related to a problem? Please describe. The fal runtime offers great easy usages like access to dbt functions (e.g. ref, source), automatically reading dbt profiles, etc.

But they are only available as a associated script to a model or project, that is, added in a schema.yml.

Describe the solution you'd like I would like to be able to run a script in the fal runtime without having to associate it to a model or project.

fal run-script my_script.py --profiles-dir <some> --project-dir <other>

And the script would have dbt functions available:

# my_script.py
ma = ref('model_a')
ma.info()

Describe alternatives you've considered A current alternative right now is to write a Python script and import fal as a package: https://docs.fal.ai/Docs/python-package

# my_script.py
from fal import FalDbt

faldbt = FalDbt(profiles_dir='<some>', project_dir='<other>')
ma = faldbt.ref('model_a')
ma.info()

and use like python my_script.py

Additional context It was asked in Discord

Is it possible to run a py script via fal without associating it in the config at all (model or pre/post hooks)? I have some ad-hoc scripts we'd like developers to execute, but they don't need to run as part of the total flow. Wondering if I can just pass it in as needed, something along the lines of fal run -s my_python_script.py.

chamini2 avatar May 18 '22 16:05 chamini2