Andrew Lamb

When we are done with the discussion, perhaps we can close this issue and use https://github.com/apache/arrow-datafusion/issues/3148 as the coordination point for development

Change this description to be a bug, which I think better reflects what is going on

> but I do think silently inferring an empty schema makes for a pretty poor UX 😅 I agree -- an error about "can not infer schema" seems much better...

> I propose that the error check we add is that at least one file exists with the specified extension. Makes sense to me

I think returning a `DataFusionError` sounds like a very reasonable thing to do When a `tokio::task` panics, it will return a `JoinError` as an error code -- perhaps somewhere datafusion...

I like this idea @Jimexist 👍 There is some related commentary / information here: https://github.com/apache/arrow-datafusion/issues/348 as well Some other crates that might be useful to consider; * `datafusion_core` (`DataFusionError`, `DFSchema`,...

Related comment: https://github.com/apache/arrow-datafusion/pull/1762#discussion_r802563462

@Jimexist welcome back! I would say that the main "break datafusion into smaller crates" has been completed. There are still some items in the description of this ticket that might...

Closing this ticket and we can track further splitting in follow on issues. Again 👏

I am thinking of trying to push us to the next level here https://github.com/apache/arrow-datafusion/issues/4181