spark-distcp icon indicating copy to clipboard operation
spark-distcp copied to clipboard

Published 20 hours ago • verified publisher icon CoxAutomotiveDataSolutions

Option to disable File checksum comparison

Open gquintana opened this issue 3 years ago • 0 comments

We would like to use spark-distcp do copy file between HDFS and S3, as the filesystem implementation is not the same sourceFS.getFileChecksum(definition.source.getPath)) and destFS.getFileChecksum(destPath) might not be the same.

We would like to be able to disable this check.

gquintana avatar Jan 18 '22 14:01 gquintana