datafusion-ballista icon indicating copy to clipboard operation
datafusion-ballista copied to clipboard

Ballista client keep blocking when prepare_task_definition or prepare_multi_task_definition fail

Open westhide opened this issue 9 months ago • 0 comments

Describe the bug Ballista client keep blocking when prepare_task_definition or prepare_multi_task_definition fail Relate to #1212

To Reproduce Steps to reproduce the behavior: Run the should_support_json_source test in #1212

    // "Error preparing task definition: Unsupported plan and extension codec failed with unsupported plan type: NdJsonExec"
    async fn should_support_json_source(
        #[future(awt)]
        #[case]
        ctx: SessionContext,
        test_data: String,
    ) {
        let result = ctx
            .read_json(&format!("{test_data}/simple.json"), Default::default())
            .await
            .unwrap()
            .collect()
            .await
            .unwrap();

        #[rustfmt::skip]
        let expected = [
            "+---+",
            "| a |",
            "+---+",
            "| 1 |",
            "+---+"
        ];

        assert_batches_eq!(expected, &result);
    }

Expected behavior should_support_json_source test should pass.

Additional context This bug cause by ballista_scheduler failed with Error preparing task definition: Unsupported plan and extension codec failed, but the job not cancel after fail, so the client keep waitting.

// "Error preparing task definition: Unsupported plan and extension codec failed with unsupported plan type: NdJsonExec"

westhide avatar Mar 22 '25 07:03 westhide