Allow "opt-out" of appending default port 9200

[erSitzt]

Hi,

we are running multiple OpenSearch Clusters in Kubernetes and most of our services access them from outside the cluster do so by using the Host-Header. This is way easier than handling the port-mapping on the ingress and i think this should at least be possible. I had the same problem before with Elasticsearch/Logstash, but i think here is a better place to take a shot at this "issue".

Using a default port, that is not clearly deductable from the URL/Config is a bad thing i think... :)

[erSitzt]

... and for OnPremise Clusters it requires additional configs on the external loadbalancers

seems very counter intuitive for something that works very easy in all other regards..

[VijayanB]

@erSitzt Thanks for creating an issue. Do you have a proposal that introduce this change without affecting existing configurations?

[erSitzt]

Hi @VijayanB , maybe an additional parameter that, when set, overrides appending the port.

as it would mostly benefit users who want to use the host header only, it could be a true/false flag defaulting to false

use_http_host_header_only => true

something like this ? i'm bad at naming stuff, but you should get the intention ;)

[VijayanB]

Hi @erSitzt , Thanks for suggestion. What do you think of creating new config called "port" with default value as 9200 and behavior as follows

1. use defaults.

The below configuration will have host as "hostname:9200"

 opensearch {
        hosts       => ["hostname"]
        user        => "admin"
        password    => "admin"
        index       => "logstash-logs-%{+YYYY.MM.dd}"
    }

2. override default port

The below configuration will have host as "hostname:1234"

 opensearch {
        hosts       => ["hostname"]
        port       => "1234"
        user        => "admin"
        password    => "admin"
        index       => "logstash-logs-%{+YYYY.MM.dd}"
    }

3. Don't append port using above apporach

The below configuration will have host as "hostname"

 opensearch {
        hosts       => ["hostname"]
        port       => ""
        user        => "admin"
        password    => "admin"
        index       => "logstash-logs-%{+YYYY.MM.dd}"
    }

[erSitzt]

It would be ok for my situation, but wouldnt it confuse people switching from ES to OpenSearch version ?

Because it's "Hosts" and not "Host" ?

the original examples were like this, so how would you handle different ports for different hosts ?

"127.0.0.1" ["127.0.0.1:9200","127.0.0.2:9200"] ["http://127.0.0.1"] ["https://127.0.0.1:9200"] ["https://127.0.0.1:9200/mypath"] (If using a proxy on a subpath)

My approach would only take affect when NO port is defined for any of the hosts and then NOT to default to 9200 to allow named based routing in a load-balancer

I think this would have less impact and cause less confusion for people familiar with logstash from Elastic

[erSitzt]

I would prefer just to be able to opt out of this default here

https://github.com/opensearch-project/logstash-output-opensearch/blob/aeee1cfbf5135fc23a6f9a2ef57ab24b7d55ca04/lib/logstash/outputs/opensearch/http_client.rb#L270

[erSitzt]

hey @VijayanB is there any progress on this ?

[brijos]

Sorry @erSitzt, the team has not had a chance to prioritize this work. We will review work across clients again in the Jul-Aug timeframe. This would be a good candidate for help wanted though so I'm adding the proper label.