datafusion-ballista
datafusion-ballista copied to clipboard
Ballista client keep blocking when prepare_task_definition or prepare_multi_task_definition fail
Describe the bug Ballista client keep blocking when prepare_task_definition or prepare_multi_task_definition fail Relate to #1212
To Reproduce
Steps to reproduce the behavior:
Run the should_support_json_source test in #1212
// "Error preparing task definition: Unsupported plan and extension codec failed with unsupported plan type: NdJsonExec"
async fn should_support_json_source(
#[future(awt)]
#[case]
ctx: SessionContext,
test_data: String,
) {
let result = ctx
.read_json(&format!("{test_data}/simple.json"), Default::default())
.await
.unwrap()
.collect()
.await
.unwrap();
#[rustfmt::skip]
let expected = [
"+---+",
"| a |",
"+---+",
"| 1 |",
"+---+"
];
assert_batches_eq!(expected, &result);
}
Expected behavior
should_support_json_source test should pass.
Additional context
This bug cause by ballista_scheduler failed with Error preparing task definition: Unsupported plan and extension codec failed, but the job not cancel after fail, so the client keep waitting.
// "Error preparing task definition: Unsupported plan and extension codec failed with unsupported plan type: NdJsonExec"