googleapis
PHP - Retry conformance in Google\Cloud\Storage\StorageClient breaks streaming large files using registerStreamWrapper
Code:
$storage = new StorageClient([
'projectId' => ...,
'keyFile' => ...,
]);
$storage->registerStreamWrapper();
$f = fopen('gs://path/to/very/large/file', 'r');
Before commit a8211f2: code runs fast (<1s, regardless file size or connection speed). $f is referencing a readable stream and can be used without fully downloading the file
After commit a8211f2: code runs slow (depending on file size and connection speed), the file is fully downloaded on the client machine. $f is referencing a readable stream from the local file
Possible cause: in vendor/google/cloud-storage/src/Connection/Rest.php:350
Utils::copyToStream($fetchedStream, $resultStream);
makes a full download of the stream, since registerStreamWrapper doesn't manage ['restOptions']['headers']['Range']
Thank you for filing this issue, and for your thorough research into the problem. I will take a look at this and see if I can't identify the issue.
@bogdanstoik I created a fix here. You can install it with composer by doing something like:
composer require google/cloud: dev-fix-storage-streamwrapper as 0.500.0
It will be in this week's release or next week, but we'd love to have you test it first to ensure it fixes your issue specifically!
I can confirm the patch fixes my specific issue. Thank you so much for the quick response!
@bogdanstoik thank YOU for notifying us of the problem and providing such a detailed issue as to make it easier to track down.