alluxio
alluxio copied to clipboard
Support the startSync feature in OSS
What changes are proposed in this pull request?
Support the startSync feature in OSS
Why are the changes needed?
By now, startSync is not supported in OSS and it throws an exception "Active Syncing is not supported on this UFS type: oss" when we execute the startSync command. The startSync feature is useful for user whose environment is Alluxio + OSS.
Does this PR introduce any user facing changes?
No, it doesn't.
Here is an example. With this startSync feature, we can use the startSync command even though the UFS is OSS. After uploading a file test3.txt to the specified OSS, we can see clearly that the metadata is updated in the Alluxio WebUI.
is there a specific version of OSS that will work with activesync, because only hdfs >2.7 had support for this..
In the implement of HDFS, it is based on iNotify. There isn't a similar way for OSS to push event to Alluxio. But in this PR, it records the last modified time of the files under the sync point, and then check and compare the last modified time of the OSS files intervally. Once it is found that the last modified time of the file is updated, the metadata of the file is going to be updated.