Issue #2574: pipe storage cp shall start data upload before traversing full source hierarchy (S3 provider)

[ekazachkova]

trafficstars

The current PR provides implementation for issue #2574

• pipe storage cp/mv operations - this feature enables for:
  • S3 provider only
  • --verify-destination shall not be enabled (since this option required full source listing)
  • --file-list shall not be enabled
• pipe storage ls operation - full listing enables for:
  • S3 provider only
  • --recursive options shall be specified
  • --show_versions shall not be specified
  • --page size limit shall not be specified
• pipe storage rm operations - added page size support for S3 provider

Note: batch size can be tined vie CP_CLI_STORAGE_LIST environment variable (default: 1000)