hamilton
hamilton copied to clipboard
stitchfix
Make `@does` better
Is your feature request related to a problem? Please describe.
Its a cool feature but its basically unusable. Current limitations:
@doesis limiting -- has to have a function of**kwargs- Type annotations are restrictive
Describe the solution you'd like
Make it so any compatible function can work. TBD on exactly how to measure compatibility but it can't be hard.
Describe alternatives you've considered Deprecate/kill it over time.
Additional context Just noodling along and decided this.
OK, feedback has suggested we keep it and invest. Thoughts about this?
import functools
import pandas as pd
from hamilton.function_modifiers import does
def _sum_series(*series: pd.Series) -> pd.Series:
return functools.reduce(series, operators.add, 0)
def _sum_series_kwargs(**kwargs) -> pd.Series:
return _sum_series(series.values())
def _sum_series_plus_intercept(intercept: float, *series: pd.Series) -> pd.Series:
return intercept + _sum_series(*series)
# provide a mapping here -- hamilton function are all kwarg so we need a *args to handle it
# This feels ugly to me, not sure we want to support *args
# Could also special-case the situation in which we have one *args in the replacing function
@does(_sum_series, series=('a', 'b', 'c'))
def a_plus_b_plus_c(a: pd.Series, b: pd.Series, c: pd.Series) -> pd.Series:
pass
@does(_sum_series_kwargs)
def a_plus_b_plus_c_with_kwargs(a: pd.Series, b: pd.Series, c: pd.Series) -> pd.Series:
pass
# This is fairly clean, IMO
@does(_sum_series_plus_intercept, series=['a', 'b', 'c'], intercept='beta')
def a_plus_b_plus_c(a: pd.Series, b: pd.Series, c: pd.Series, beta: float) -> pd.Series:
pass
def _load_from_table(table: str, db: str) -> pd.DataFrame:
pass
# Loads from a table, knows about table/db
@does(_load_from_table)
def my_data(table: str, db: str) -> pd.DataFrame:
pass
# we have different keyword arguments than the function initially, we have to map them
# I like this but its confusing -- not 100% clear that `table` maps to the argument `foo_table` rather than the vlaue `foo_table`
@does(_load_from_table, table='foo_table', db='bar_db')
def foo_bar_data(foo_table: str, bar_db: str) -> pd.DataFrame:
pass