Andrew Lamb
Andrew Lamb
> fix the issue properly I don't know what you mean by 'fix it properly" -- do you mean somehow have an API in polars that provides credentials via arbitrary...
BTW here is an example of how we use the AWS SKD to get credentials in datafusion-cli: https://github.com/apache/datafusion/blob/747001a41481e0cf39dc758a85d1bdb64fdeb7c0/datafusion-cli/src/object_storage.rs#L65-L133
I plan to review this tomorrow morning
cc @phillipleblanc as I think you may be interested in this too
An also related conversation maybe: https://github.com/apache/datafusion/issues/10534
I am working on an alternate proposal too -- stay tuned
Here is an alternate idea https://github.com/apache/datafusion/pull/11168
@jayzhan211's alternate PR here https://github.com/apache/datafusion/pull/11180 is looking good in my opinion
> We can consider implementing a `LIMIT` either for the approach in this PR, or for a possible physical plan operator. I could work on an operator implementation if this...
Maybe @robtandy could help