Antoine Pitrou
Antoine Pitrou
We're not supposed to break the ABI in bugfix releases, so I don't know why the soversion was bumped.
Ah, right. Well, 16.1.0 is a slightly unusual case then.
> Previously, we would fail when opening the output stream if the bucket doesn't exist. We inferred that by sending the `CreateMultipartUpload` request, which we now do not send anymore...
My point is that _if the path cannot be written to_, the error happens when opening the file, not later on.
> That is true. I guess the question is if `arrow`'s OutputStream API makes an explicit _guarantee_ that `Open` should throw if the target does not exist. My guess would...
> We can do that. I would propose that if the optimization is disabled, we directly use multi-part uploads (basically replicating the old behavior). That sounds reasonable to me.
@github-actions crossbow submit -g cpp -g python
@github-actions crossbow submit -g cpp -g python
@mapleFU I didn't know this was possible. This looks neat in the principle. Did you try to run some benchmark?
Perhaps you can try with the additional benchmarks in https://github.com/apache/arrow/pull/41761