chore(warehouse): validate object storage credentials

[deepakrai9185720]

trafficstars

Description

We store events that are flowing through our systems so that in case of any issues, we can replay them back and get at correct state. We store these events in bucket owned by rudderstack.

Going forward, we will be asking customers if they want to store the events on there own systems and if yes, allow them to add the credentials of the storage. The task is to expose a validation endpoint in warehouse to capture the creds and validate them on account of ability to upload and download.

The outcome of this validation will be presented to the user.

Notion Ticket

https://www.notion.so/rudderstacks/Object-Storage-Validation-Design-0a3300c4754d4271bed53d9ad15d683d

Security

• [x] The code changed/added as part of this pull request won't create any security issues with how the software is being used.

[codecov[bot]]

Codecov Report

Base: 44.25% // Head: 45.14% // Increases project coverage by +0.89% :tada:

Coverage data is based on head (20a6402) compared to base (73cc48a). Patch coverage: 29.28% of modified lines in pull request are covered.

:exclamation: Current head 20a6402 differs from pull request most recent head b524789. Consider uploading reports for the commit b524789 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2440      +/-   ##
==========================================
+ Coverage   44.25%   45.14%   +0.89%     
==========================================
  Files         181      178       -3     
  Lines       38413    37777     -636     
==========================================
+ Hits        17000    17056      +56     
+ Misses      20339    19643     -696     
- Partials     1074     1078       +4     

Impacted Files	Coverage Δ
services/filemanager/filemanager.go	53.09% <0.00%> (-1.45%)	:arrow_down:
warehouse/errors.go	0.00% <0.00%> (ø)
warehouse/api.go	70.41% <26.50%> (-6.44%)	:arrow_down:
warehouse/warehousegrpc.go	73.43% <38.00%> (-22.72%)	:arrow_down:
warehouse/schema.go	47.82% <0.00%> (-1.16%)	:arrow_down:
jobsdb/jobsdb.go	67.75% <0.00%> (-0.10%)	:arrow_down:
warehouse/utils/utils.go	58.83% <0.00%> (ø)
warehouse/bigquery/bigquery.go	0.00% <0.00%> (ø)
warehouse/postgres/postgres.go	0.00% <0.00%> (ø)
warehouse/redshift/redshift.go	0.00% <0.00%> (ø)
... and 16 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[achettyiitr]

Depending upon the rudder server features settings. https://github.com/rudderlabs/rudder-server/pull/2377