dagster icon indicating copy to clipboard operation
dagster copied to clipboard

Resolve and validate input config when directly invoking IO manager and root manager methods

Open sryza opened this issue 2 years ago • 0 comments

Reported here: https://dagster.slack.com/archives/C01U954MEER/p1649685707570159

@root_input_manager(
    input_config_schema={
        "filepath": Field(str, is_required=True),
        "sep": Field(str, default_value=",", is_required=False, description="Delimiter to use."),
    }
)
def csv_to_df(context):
    return pd.read_csv(
        context.config["filepath"],
        sep=context.config.get("sep")
    )

In this test, the default value for sep is not applied:

    def test_csv_to_df(self):
        manager = csv_to_df(None)
        context = build_input_context(
            config={"filepath": os.path.join(self.fixtures_dir, "csv_to_df.csv")}
        )

        df = manager.load_input(context)
        ...

This is inconsistent with how we handle direct invocation of ops, for which we do config resolution and validation.

sryza avatar Apr 11 '22 15:04 sryza