jf

Results 21 comments of jf

These two functionalities achieve the same thing ```python %%cell_to_module -m pipe_output_with_target --display from typing import Dict from hamilton.function_modifiers import extract_fields, pipe_input, pipe_output, step def _pre_step(something:int)->int: return something + 10 def...

> > These two functionalities achieve the same thing > > ```python > > %%cell_to_module -m pipe_output_with_target --display > > from typing import Dict > > from hamilton.function_modifiers import extract_fields,...

> 2. Worth documenting assumptions -- the code is clean but there are some clever things (E.G. the `Applicable` stuff) that we want to make sure is clear as to...

> Yes more dev --> but the idea is that we can provide prod introspection, so ideally you wouldn't need to change things. So I wouldn't expect people to translate...

> > Right -- but the following should work though right? > > ```python > @mutate(foo) > def _normalize(df: pd.DataFrame) -> pd.DataFrame: > ... > > @mutate(foo) > def _something_funky(df:...

> > Something to think about/a caveat: > > > > 1. If `@mutate` runs once it's good > > 2. If you run that again, it might double-up >...

![Screenshot 2024-10-12 at 11 12 33](https://github.com/user-attachments/assets/82d03c32-5cad-4454-95b7-796f8c557a50) I just checked, `pipe_output`, `pipe_input` / `step` has the same behavior (not surprised)!

> The problem here is the originating functions... `@mutate` isn't attached to the function and we don't collect the references. We may want to: > > 1. Add to originating...

> > @jernejfrank any thoughts on what assumption I broke with `@mutate`? > > figured it out. I have a hack that works I think. will need to refactor a...

@skrawcz ah, okay - didn't know about RayTaskExecutor and haven't had the time/need to touch the parallel module. Our pipelines are pretty split up and only one of them uses...