dagster icon indicating copy to clipboard operation
dagster copied to clipboard

SourceAssets don't support InputManagers

Open danielgafni opened this issue 2 years ago • 6 comments

Dagster version

1.2.2

What's the issue?

Why does SourceAsset take io_manager_def or io_manager_key as input, but doesn't support input_manager_def or input_manager_key? I'm writing InputManagers specifically for my SourceAssets, but passing them creates typing errors. Doesn't seem right to me

dagster._check.ParameterCheckError: Param "io_manager_def" is not a IOManagerDefinition. Got <dagster._core.storage.input_manager.InputManagerDefinition object at 0x7fc9f0028e80> which is type <class 'dagster._core.storage.input_manager.InputManagerDefinition'>.

What did you expect to happen?

SourceAssets should support InputManagers since they can only be loaded, not materialized

How to reproduce?

No response

Deployment type

Other

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

danielgafni avatar Mar 16 '23 17:03 danielgafni

Hey! Thanks for reporting this.

@smackesey would you be the right person to give a heads-up about this?

tacastillo avatar Mar 16 '23 17:03 tacastillo

yes I'll investigate

smackesey avatar Mar 16 '23 18:03 smackesey

@smackesey any plans to address this?

I was looking at using the InputManager since it seems would go greatly hand in hand with the concept of a source asset, since you only need to define Handle Input.

Obviously I can do this with a custom IO manager that does pass in handle_output but I would prefer to use the Input Manager for this..

ion-elgreco avatar Feb 24 '24 14:02 ion-elgreco

@smackesey any plans to address this?

Looking today.

smackesey avatar Mar 19 '24 17:03 smackesey

Possibly related issue: #21532

czroth avatar May 01 '24 16:05 czroth

@smackesey any plans to address this?

Looking today.

No success, I guess?

ion-elgreco avatar May 04 '24 18:05 ion-elgreco