Core: reduce scale factor for HadoopFileIOTest prefix tests

[jackye1995]

When running these tests, I notice it's taking a long time locally. I tried a few combinations

Current scale factors: 2 min 25 seconds scale factors (1, 10, 1000): 46 seconds scale factors (1, 5, 500): 24 seconds

Could we reduce the scale factor to (1, 5, 500)? It seems sufficient to me for the operation to run for half a minute for testing purpose. Is there anything that we need to test only in 2500 scale factor?

@edgarRd @danielcweeks

[nastra]

I think it's fine reducing those scale factors, but it's quite interesting that this takes so long for you. I've ran HadoopFileIOTest a few times locally and it takes about 18 secs in total:

[danielcweeks]

@jackye1995 The reason for the scale factor was to trigger the list paging behavior and test the boundaries. That doesn't happen if we reduce it to 500. We could make the max results for list configurable, but that seems overcomplicated (unless we can do it just in the test case and not have to plumb the properties through and further complicate AwsProperties.

[github-actions[bot]]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.