HDDS-7178. [Multi-Tenant] Use optimistic read in Ranger background sync

[errose28]

What changes were proposed in this pull request?

Coordination between the background sync and tenant modification requests is currently done through a stamped lock. The background sync currently holds a read lock for a short period every time it runs to fetch a consistent view of OM and Ranger state, however, tenant modifications will be blocked during this time since they need the write lock.

Since this is already a stamped lock, we can use optimistic read with retries instead of a read lock in the background sync. This way it will not conflict with user tenant modifications as long as there is no divergence between OM and Ranger. If there is divergence, the background sync will still need to push updates under a write lock, blocking tenant modifications while it does so.

This PR also adds some logging improvements to the background sync.

What is the link to the Apache JIRA

HDDS-7178

How was this patch tested?

• [x] Unit test added
• [ ] Test background sync with live requests and a real Ranger instance (TODO)

[errose28]

@smengcl @prashantpogde PTAL when you get a chance for feedback on this approach. This is pending testing with live Ranger as well. If we want to change the default frequency with which the background sync runs (it's currently every 10 minutes) we can do that in this PR too.

[smengcl]

@errose28 Would you resolve the conflict and address the comments?

[DaveTeng0]

hey @errose28 ~ the CI build step of findbugs failed~ please take a look when you have some time. thanks!

[kerneltime]

@errose28 can you please rebase?

[adoroszlai]

Thanks @errose28 for the improvement, @prashantpogde, @smengcl for the review.