hamilton icon indicating copy to clipboard operation
hamilton copied to clipboard

Can't stack @inject and @parameterize decorators

Open j7zAhU opened this issue 1 year ago • 3 comments

As per the issue title. This throws an error:

from hamilton.function_modifiers import inject, source, parameterize

@inject(params=source('my_func__params'))
@parameterize(
    my_func_a={'date_range': source('my_func_a_date_range')},
    my_func_b={'date_range': source('my_func_b_date_range')}
)
def my_func(date_range: tuple[pd.Timestamp, pd.Timestamp], params: int) -> int:
    return 1

temp_module = ad_hoc_utils.create_temporary_module(
    my_func, module_name="my_module"
)

config = {
    'my_func__params': 1,
    'my_func_a__date_range': [pd.Timestamp('2022-01-01'), pd.Timestamp('2022-01-10')],
    'my_func_b__date_range': [pd.Timestamp('2023-01-01'), pd.Timestamp('2023-01-10')],
}
dr = driver.Driver(config, temp_module)
df = dr.execute(final_vars=['my_func_a'])

Library & System Information

Hamilton: 1.22.5 , Python 3.11.1

j7zAhU avatar May 04 '23 13:05 j7zAhU

OK, so no inherent reason why this should work -- its an implementation detail that it doesn't. That said, the implementation is a little complex, and the more "hamiltonian" way is one of the following:

  1. (cleanest IMO) -- actually make the source the name in the function (or if its a value make it the default value)
  2. (slightly repetitive) -- integrate it with the @parameterize

*** details below ***

The reason this doesn't work is because @inject is implemented as another parameterization with one value. This is not the cleanest way -- instead it should be able to inject a node into the subdag/replace a dependency within the subdag on something specific. I think this could pretty easily be done, yet it requires a small refactor. Will dig in at some point soon, but for now I think there are a few unblocking strategies here.

As discussed on slack, using group might be helpful as well.

elijahbenizzy avatar May 04 '23 17:05 elijahbenizzy

I would like to work on this Bug! Can i get the path of the file for this code please

149189 avatar Sep 29 '23 18:09 149189

I would like to work on this Bug! Can i get the path of the file for this code please

Apologies @149189. We never got back to you here.

@elijahbenizzy this feels similar to https://github.com/DAGWorks-Inc/hamilton/issues/748 right?

skrawcz avatar Jul 18 '24 19:07 skrawcz