hadoop
hadoop copied to clipboard
HADOOP-19140. [ABFS, S3A] Add IORateLimiter API
Adds an API (pulled from #6596) to allow callers to request IO capacity for an named operation with optional source and dest paths.
The first use of this would be the bulk delete operation of #6494; there'd be some throttling within the s3a code which set max # of writes per bucket and for the bulk delete the caller would ask for as many as there were entries.
Added new store operations for delete_bulk and delete_dir
How was this patch tested?
New tests.
For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
- [ ] If applicable, have you updated the
LICENSE
,LICENSE-binary
,NOTICE-binary
files?