feat: allow to set a signature port for tunnel usage

[andreaslang]

Description

Allows to change the port used to sign the AWS request which is causing an issue if accessing an AWS Opensearch instance via a tunnel

If you have an ssh tunnel created this works now (while it would not without signature_port):

host = 'localhost'
port = 10012
region = 'eu-west-1'
service = 'es'
credentials = boto3.Session().get_credentials()
auth = AWSV4SignerAuth(credentials, region, service, signature_port=443)

client = OpenSearch(
    hosts = [{'host': host, 'port': port}],
    http_auth = auth,
    use_ssl = True,
    verify_certs = False,
    ssl_assert_hostname = False,
    ssl_show_warn = False,
    connection_class = RequestsHttpConnection,
    pool_maxsize = 20
)

Issues Resolved

https://github.com/opensearch-project/opensearch-py/issues/184

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 70.58%. Comparing base (c8b04a5) to head (4e3ef98). Report is 87 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #491      +/-   ##
==========================================
- Coverage   70.92%   70.58%   -0.34%     
==========================================
  Files          81       81              
  Lines        7732     7738       +6     
==========================================
- Hits         5484     5462      -22     
- Misses       2248     2276      +28     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[dblock]

I would merge a change that allows to override headers, and specifically to override the host header.

[andreaslang]

I would merge a change that allows to override headers, and specifically to override the host header.

Yes, that is a really good point. It should work the same way if I replace the host header like localhost:10012 > realdomain:443. I will change the PR having an Authentication class specifically for that (e.g. TunneledAWSV4SignerAuth) with the explanation in the docstring. Will likely get to it next Tuesday.

[dblock]

I would merge a change that allows to override headers, and specifically to override the host header.

Yes, that is a really good point. It should work the same way if I replace the host header like localhost:10012 > realdomain:443. I will change the PR having an Authentication class specifically for that (e.g. TunneledAWSV4SignerAuth) with the explanation in the docstring. Will likely get to it next Tuesday.

I actually would prefer as a developer to be able to write something like this:

client = OpenSearch(
    hosts = [{'host': host, 'port': port}],
    http_headers: {
        'Host': 'something.us-west2.aws.bla.bla.bla'
    }
)

The signer implementation should be smart enough to consider this host header.

This seems more generic and future-proof, don't you think?

[dblock]

@andreaslang Are you still interested in adding an http_headers option?

[andreaslang]

Yes, I am. Sorry for the delay. I do have a good excuse that my daughter was born though. Planning to pick this up soon.

On Thu, 9 Nov 2023, 23:40 Daniel (dB.) Doubrovkine, < @.***> wrote:

@andreaslang https://github.com/andreaslang Are you still interested in adding an http_headers option?

— Reply to this email directly, view it on GitHub https://github.com/opensearch-project/opensearch-py/pull/491#issuecomment-1804848291, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZJ2U3FVZSYEZW7CZ7R55LYDVSVDAVCNFSM6AAAAAA4UZOLR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBUHA2DQMRZGE . You are receiving this because you were mentioned.Message ID: @.***>

[saimedhi]

Hello @andreaslang! Hope you're doing great. Could you please consider finishing up this PR when you have a moment? Thank you!