Steve Loughran

merged to trunk. @anujmodi2021 can you do a PR and retest with branch -then I will merge it there too.

Just going to highlight that the contract test failed *badly* when reading into direct buffers from azure. I think this is a bug in the direct buffer fetching logic -it's...

may need an addendum here; seeing problems on 3.4 backport *and* I left out the patch number in the commit message. lets see what happens on trunk with/without prefetch enabled

ok, trunk fails too. going to revert; must have failed to do that final run. will have to do an addendum pr

@shameersss1 what do you think here? actually, maybe under magic paths we skip trying to create dirs at all, at least on the in-memory mode. no files to look for...

reviewed, i'm just wondering how to make the test the cleanest. Going to invite reviews from @shameersss1 @ahmarsuhail @HarshitGupta11 @mukund-thakur as they've been looking around here. Does anyone expect anything...

Don't think I have forgotten about this -I have just been very distracted. I'm wondering if we should provide a list of options to "optimise", e.g "create, mkdir, delete" and...

I'm going to propose we change how the options are done, and do something similar for ABFS. I think we need something like C/C++ optimisers where you pass in a...

I'm doing a quick PR of the design; @HarshitGupta11 and I discussed it.

> Got it, i was planning to embed the logic as part of this PR sometime early next week but separate PR sounds more manageable! it's a bit blurred as...