hashicorp
Ingress Gateway: no routing, 404 Not Found
Hi Team!
Overview of the Issue
I'm trying to setup ingress gateway with routing based on Hosts field (or any simple setup, including wildcard on Name field).
Envoy always returns 404 Not Found
Reproduction Steps
- Create ingress gw. using this config:
"Kind": "ingress-gateway",
"Name": "us-east-ingress",
"TLS": {
"Enabled": false
},
"Listeners": [
{
"Port": 80,
"Protocol": "http",
"Services": [
{
"Name": "pk-server",
"Hosts": [
"pk-api.example.com",
"pk-server.ingress.consul",
"*.consul"
]
}
]
}
],
"CreateIndex": 130475,
"ModifyIndex": 131469
}
- Register consul service with connect proxy
{
"service": {
"port": 5002,
"name": "pk-server",
"connect": { "sidecar_service": {} }
}
}
- Set proxy-defaults:
Kind = "proxy-defaults"
Name = "global"
Config {
protocol = "http"
}
-
Run connect proxy:
consul connect envoy -gateway=ingress -register -service='us-east-ingress' -address 'xx.xx.xx.xx:80' -token='<consul_token>' -admin-bind='xx.xx.xx.xx:1900' -omit-deprecated-tags -
Try curl to the service:
curl -v -H "Host: pk-server.ingress.consul" <private_ip>
* Trying xx.xx.xx.xx:80...
* TCP_NODELAY set
* Connected to xx.xx.xx.xx (xx.xx.xx.xx) port 80 (#0)
> GET / HTTP/1.1
> Host: pk-server.ingress.consul
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< date: Wed, 09 Dec 2020 14:26:12 GMT
< server: envoy
< content-length: 0
<
* Connection #0 to host xx.xx.xx.xx left intact
curl using any value from Hosts also failing.
Debugging
Name resolution looks good:
dig +short @127.0.0.1 -p 8600 pk-server.ingress.consul SRV
1 1 80 0af00037.addr.pk.consul.
connect intentions
[
{
"CreatedAt": "2020-12-09T15:45:12.752819454Z",
"UpdatedAt": "2020-12-09T15:45:12.752819454Z",
"ID": "ff656e3b-cdb8-cbbd-fb53-c875a8895564",
"SourceNS": "default",
"SourceName": "us-east-ingress",
"DestinationNS": "default",
"DestinationName": "pk-server",
"SourceType": "consul",
"Action": "allow",
"Precedence": 9,
"Hash": "sY36OcnUwSTzuD6265Sl5fyWZRIvD4zKC4k84Duk3rI=",
"CreateIndex": 132413,
"ModifyIndex": 132413
},
{
"CreatedAt": "2020-11-30T13:38:55.49354711Z",
"UpdatedAt": "2020-11-30T13:38:55.49354711Z",
"ID": "e46aeb04-1fb6-b002-0985-044169c0bb87",
"Description": "all-1-all",
"SourceNS": "default",
"SourceName": "*",
"DestinationNS": "default",
"DestinationName": "*",
"SourceType": "consul",
"Action": "allow",
"Precedence": 5,
"Hash": "kYyv7O318vAEgLeAeILxm32zxfYX5bpKLJSgH7Ze5xs=",
"CreateIndex": 279,
"ModifyIndex": 279
}
]
services associated with an ingress gateway
[
{
"Gateway": {
"Name": "us-east-ingress"
},
"Service": {
"Name": "pk-server"
},
"GatewayKind": "ingress-gateway",
"Port": 80,
"Protocol": "http",
"Hosts": [
"pk-api.stobworg.co",
"pk-server.ingress.consul",
"*.consul"
],
"CreateIndex": 132076,
"ModifyIndex": 132076
}
]
envoy dynamic route config
"dynamic_route_configs": [
{
"version_info": "00000001",
"route_config": {
"@type": "type.googleapis.com/envoy.api.v2.RouteConfiguration",
"name": "80",
"virtual_hosts": [
{
"name": "pk-server",
"domains": [
"pk-api.example.com",
"pk-server.ingress.consul",
"*.consul",
"pk-api.example.com:80",
"pk-server.ingress.consul:80",
"*.consul:80"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "pk-server.default.pk.internal.fa8428e8-a896-b606-3983-953731abba55.consul"
}
}
]
}
],
"validate_clusters": true
},
"last_updated": "2020-12-09T14:45:12.238Z"
}
]
}
Consul info for both Client and Server
Client and server info
agent:
check_monitors = 0
check_ttls = 0
checks = 8
services = 8
build:
prerelease =
revision = a417fe51
version = 1.9.0
consul:
acl = enabled
bootstrap = false
known_datacenters = 1
leader = false
leader_addr = xx.xx.xx.xx:8300
server = true
raft:
applied_index = 131326
commit_index = 131326
fsm_pending = 0
last_contact = 38.86848ms
last_log_index = 131326
last_log_term = 6
last_snapshot_index = 131095
last_snapshot_term = 6
latest_configuration = [{Suffrage:Voter ID:703fc5a4-6520-f587-82f8-26c75c49d508 Address:xx.xx.xx.xx:8300} {Suffrage:Voter ID:fc705b76-eebe-d5b0-6670-876eabdfd6a4 Address:xx.xx.xx.xx:8300} {Suffrage:Voter ID:f8e3faf6-da8f-e021-df69-5f8729aaa584 Address:xx.xx.xx.xx:8300}]
latest_configuration_index = 0
num_peers = 2
protocol_version = 3
protocol_version_max = 3
protocol_version_min = 0
snapshot_version_max = 1
snapshot_version_min = 0
state = Follower
term = 6
runtime:
arch = amd64
cpu_count = 1
goroutines = 202
max_procs = 1
os = linux
version = go1.15.5
serf_lan:
coordinate_resets = 0
encrypted = true
event_queue = 0
event_time = 6
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 11
members = 4
query_queue = 0
query_time = 1
serf_wan:
coordinate_resets = 0
encrypted = true
event_queue = 0
event_time = 1
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 12
members = 3
query_queue = 0
query_time = 1
Operating system and Environment details
OS
Distributor ID: Ubuntu
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal
envoy
envoy version: 8fb3cb86082b17144a80402f5367ae65f06083bd/1.16.0/clean-getenvoy-a5345f6-envoy/RELEASE/BoringSSL
Consul
Consul v1.9.0
Revision a417fe510
@pkrolikowski / @HofmannZ - Did you find a solution? I am facing similar issue with HTTP routing. Any work around will also be helpful
@neonthe1way yes, in our case we needed to add the correct host headers on the request so that the ingress gateway knew where to route to.
In the end we never ended up using it in production tho.
@pkrolikowski / @HofmannZ - Did you find a solution? I am facing similar issue with HTTP routing. Any work around will also be helpful
As workaround you can use wildcard for Hosts header, which accept all its values like:
//...
Services = [
{
Name = "public-api"
Hosts = ["*"]
}
]
