andersundsehr
Do not iterate over all files if number of items is given
to avoid a long running process the foreach loop in aus_driver_amazon_s3/Classes/Driver/AmazonS3Driver.php:850
should consider the $numberOfItems.
If you've a bucket with a lot of files, it can take a lot of time to read them all, even if you just want to get a specific number of files.
Maybe the \array_splice from line /aus_driver_amazon_s3/Classes/Driver/AmazonS3Driver.php:877 cacn/should be done before the foreach.
Example:
if ($response['Contents']) {
if ($numberOfItems > 0) {
$response['Contents'] = array_splice($response['Contents'], $start, $numberOfItems);
}
foreach ($response['Contents'] as $fileCandidate) {
// the rest of the code
Link to the code-line mentioned above for easier access: https://github.com/andersundsehr/aus_driver_amazon_s3/blob/master/Classes/Driver/AmazonS3Driver.php#L855
@dknx01
The code in that loop does not do much. But it does a filtering on the elements, i think if we slice before the filters, the results get wrong.
Anyway, meanwhile some improvements were done, in special "CAPABILITY_HIERARCHICAL_IDENTIFIERS" was added to the extension which fastens up many processes by using a shortcut in the database.
For TYPO3 v11 there is a open patch https://review.typo3.org/c/Packages/TYPO3.CMS/+/79444 For aus_driver_amazon_s3 the latest improvements are in Branch feature/performance-improvements-caching but need some further testing before release.
