dagster
dagster copied to clipboard
Resolve and validate input config when directly invoking IO manager and root manager methods
Reported here: https://dagster.slack.com/archives/C01U954MEER/p1649685707570159
@root_input_manager(
input_config_schema={
"filepath": Field(str, is_required=True),
"sep": Field(str, default_value=",", is_required=False, description="Delimiter to use."),
}
)
def csv_to_df(context):
return pd.read_csv(
context.config["filepath"],
sep=context.config.get("sep")
)
In this test, the default value for sep is not applied:
def test_csv_to_df(self):
manager = csv_to_df(None)
context = build_input_context(
config={"filepath": os.path.join(self.fixtures_dir, "csv_to_df.csv")}
)
df = manager.load_input(context)
...
This is inconsistent with how we handle direct invocation of ops, for which we do config resolution and validation.