lakectl - HTTP Client Supporting Retries

Open eladlachmi opened this issue 10 months ago • 2 comments

Related to #7705

Change Description

Background

See the associated issue for context

New Feature

This PR is only part of the implementation of #7705, but I've decided this part is reviewable and mergeable as a standalone unit. Hopefully, this will make this and other PRs of this implementation easier to reason about and review.

This PR creates a new HTTP client for use in lakectl, based on go-retryablehttp. It supports retries with a custom retry policy tailored for lakeFS and an exponential backoff retry strategy. Its constructor func includes several parameters to tune the retry max count and intervals and the option to provide a custom http.Transport. For seamless integration later on the constructor returns a standard *http.Client.

Testing Details

This PR includes a suite of tests for the custom retry policy and a single test of the client itself with an httptest.Server. I'm not sure if client tests add any value, or maybe we're just retesting things that are already covered by tests in the source repository. Would like your feedback on this.
If we decide they add some additional value, I can create a few more for better coverage.