emissary icon indicating copy to clipboard operation
emissary copied to clipboard

Published 20 hours ago • verified publisher icon emissary-ingress

Emissary cannot be used as the network layer of knative

Open wanjunlei opened this issue 3 years ago • 4 comments

Describe the bug Using emissary 2.1.0 as the network layer of knative 1.0.1, both emissary and knative are running normally. Environment variable AMBASSADOR_KNATIVE_SUPPORT is set. The ingress.class of knative had changed to ambassador.ingress.networking.knative.dev.

Create a knative serving, emissary cannot update the status of ingress. The log of emissary is this.

2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: TIMER reconfiguration: 8, 0.074/0.124/0.220

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: TIMER Fetcher: 8, 0.050/0.078/0.118

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: TIMER AConf: 8, 0.002/0.010/0.065

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: TIMER IR: 8, 0.006/0.016/0.063

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: TIMER EConf: 8, 0.001/0.002/0.004

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: Total requests: 25

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: Hit ratio:      8.0%

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: Invalidations:  0 calls

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE:                 0 objects

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: complete reconfigure at 2022-01-24T03:44:00.872558

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incremental reconfigure at 2022-01-24T03:47:24.430081

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incremental reconfigure at 2022-01-24T03:47:24.678660

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incremental reconfigure at 2022-01-24T03:47:27.678575

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incremental reconfigure at 2022-01-24T03:47:27.794485

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incremental reconfigure at 2022-01-24T03:47:31.774669

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incremental reconfigure at 2022-01-24T03:47:31.974189

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incremental reconfigure at 2022-01-24T03:49:29.551364

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incremental count: 7

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: complete count: 1

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incrementals outstanding: 7

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: incremental checks: 0, errors 0

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: last_complete 2022-01-24T03:44:00.872558

 2022-01-24 03:50:02 diagd 2.1.0 [P23TAEW] INFO: CACHE: last_check 2022-01-24T03:44:00.872558

 2022-01-24 03:50:15 diagd 2.1.0 [P23TAEW] INFO: Updating Knative Ingress serving-w6rz4-ksvc-zkdj6 status to ('ingress.networking.internal.knative.dev', 'default', {'observedGeneration': 1, 'conditions': [{'lastTransitionTime': '2022-01-24T03:50:15Z', 'status': 'True', 'type': 'LoadBalancerReady'}, {'lastTransitionTime': '2022-01-24T03:50:15Z', 'status': 'True', 'type': 'NetworkConfigured'}, {'lastTransitionTime': '2022-01-24T03:50:15Z', 'status': 'True', 'type': 'Ready'}], 'loadBalancer': {'ingress': [{'domainInternal': 'emissary-ingress.emissary.svc.cluster.local'}]}, 'privateLoadBalancer': {'ingress': [{'domainInternal': 'emissary-ingress.emissary.svc.cluster.local'}]}})

 2022-01-24 03:50:15 diagd 2.1.0 [P23TAEW] INFO: EnvoyConfig: Generating V3

 2022-01-24 03:50:15 diagd 2.1.0 [P23TAEW] WARNING: No active listeners at all; check your Listener and Host configuration

 2022-01-24 03:50:15 diagd 2.1.0 [P23TAEW] INFO: configuration updated (incremental) from snapshot snapshot (S48 L0 G7 C2)

 time="2022-01-24 03:50:15.4183" level=info msg="Loaded file /ambassador/envoy/envoy.json" func=github.com/datawire/ambassador/v2/cmd/ambex.Decode file="/go/cmd/ambex/main.go:279" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:15.4188" level=info msg="Saved snapshot v9" func=github.com/datawire/ambassador/v2/cmd/ambex.csDump file="/go/cmd/ambex/main.go:369" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:15.4195" level=info msg="Pushing snapshot v9" func=github.com/datawire/ambassador/v2/cmd/ambex.updaterWithTicker file="/go/cmd/ambex/ratelimit.go:159" CMD=entrypoint PID=1 THREAD=/ambex

 2022-01-24 03:50:22 diagd 2.1.0 [P23TAEW] INFO: Updating Knative Ingress serving-w6rz4-ksvc-zkdj6 status to ('ingress.networking.internal.knative.dev', 'default', {'observedGeneration': 1, 'conditions': [{'lastTransitionTime': '2022-01-24T03:50:22Z', 'status': 'True', 'type': 'LoadBalancerReady'}, {'lastTransitionTime': '2022-01-24T03:50:22Z', 'status': 'True', 'type': 'NetworkConfigured'}, {'lastTransitionTime': '2022-01-24T03:50:22Z', 'status': 'True', 'type': 'Ready'}], 'loadBalancer': {'ingress': [{'domainInternal': 'emissary-ingress.emissary.svc.cluster.local'}]}, 'privateLoadBalancer': {'ingress': [{'domainInternal': 'emissary-ingress.emissary.svc.cluster.local'}]}})

 2022-01-24 03:50:22 diagd 2.1.0 [P23TAEW] INFO: EnvoyConfig: Generating V3

 2022-01-24 03:50:22 diagd 2.1.0 [P23TAEW] WARNING: No active listeners at all; check your Listener and Host configuration

 2022-01-24 03:50:22 diagd 2.1.0 [P23TAEW] INFO: configuration updated (incremental) from snapshot snapshot (S47 L0 G7 C2)

 time="2022-01-24 03:50:22.1652" level=info msg="Loaded file /ambassador/envoy/envoy.json" func=github.com/datawire/ambassador/v2/cmd/ambex.Decode file="/go/cmd/ambex/main.go:279" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:22.1655" level=info msg="Saved snapshot v10" func=github.com/datawire/ambassador/v2/cmd/ambex.csDump file="/go/cmd/ambex/main.go:369" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:22.1659" level=info msg="Pushing snapshot v10" func=github.com/datawire/ambassador/v2/cmd/ambex.updaterWithTicker file="/go/cmd/ambex/ratelimit.go:159" CMD=entrypoint PID=1 THREAD=/ambex

 2022-01-24 03:50:22 diagd 2.1.0 [P23TAEW] INFO: EnvoyConfig: Generating V3

 2022-01-24 03:50:22 diagd 2.1.0 [P23TAEW] WARNING: No active listeners at all; check your Listener and Host configuration

 2022-01-24 03:50:22 diagd 2.1.0 [P23TAEW] INFO: configuration updated (incremental) from snapshot snapshot (S45 L0 G3 C1)

 time="2022-01-24 03:50:22.3603" level=info msg="Loaded file /ambassador/envoy/envoy.json" func=github.com/datawire/ambassador/v2/cmd/ambex.Decode file="/go/cmd/ambex/main.go:279" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:22.3606" level=info msg="Saved snapshot v11" func=github.com/datawire/ambassador/v2/cmd/ambex.csDump file="/go/cmd/ambex/main.go:369" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:22.3610" level=info msg="Pushing snapshot v11" func=github.com/datawire/ambassador/v2/cmd/ambex.updaterWithTicker file="/go/cmd/ambex/ratelimit.go:159" CMD=entrypoint PID=1 THREAD=/ambex

 2022-01-24 03:50:25 diagd 2.1.0 [P23TAEW] INFO: EnvoyConfig: Generating V3

 2022-01-24 03:50:25 diagd 2.1.0 [P23TAEW] WARNING: No active listeners at all; check your Listener and Host configuration

 time="2022-01-24 03:50:25.3619" level=info msg="Loaded file /ambassador/envoy/envoy.json" func=github.com/datawire/ambassador/v2/cmd/ambex.Decode file="/go/cmd/ambex/main.go:279" CMD=entrypoint PID=1 THREAD=/ambex

 2022-01-24 03:50:25 diagd 2.1.0 [P23TAEW] INFO: configuration updated (incremental) from snapshot snapshot (S46 L0 G3 C1)

 time="2022-01-24 03:50:25.3623" level=info msg="Saved snapshot v12" func=github.com/datawire/ambassador/v2/cmd/ambex.csDump file="/go/cmd/ambex/main.go:369" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:25.3631" level=info msg="Pushing snapshot v12" func=github.com/datawire/ambassador/v2/cmd/ambex.updaterWithTicker file="/go/cmd/ambex/ratelimit.go:159" CMD=entrypoint PID=1 THREAD=/ambex

 2022-01-24 03:50:25 diagd 2.1.0 [P23TAEW] INFO: EnvoyConfig: Generating V3

 2022-01-24 03:50:25 diagd 2.1.0 [P23TAEW] WARNING: No active listeners at all; check your Listener and Host configuration

 2022-01-24 03:50:25 diagd 2.1.0 [P23TAEW] INFO: configuration updated (incremental) from snapshot snapshot (S47 L0 G3 C1)

 time="2022-01-24 03:50:25.4455" level=info msg="Loaded file /ambassador/envoy/envoy.json" func=github.com/datawire/ambassador/v2/cmd/ambex.Decode file="/go/cmd/ambex/main.go:279" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:25.4460" level=info msg="Saved snapshot v13" func=github.com/datawire/ambassador/v2/cmd/ambex.csDump file="/go/cmd/ambex/main.go:369" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:25.4465" level=info msg="Pushing snapshot v13" func=github.com/datawire/ambassador/v2/cmd/ambex.updaterWithTicker file="/go/cmd/ambex/ratelimit.go:159" CMD=entrypoint PID=1 THREAD=/ambex

 2022-01-24 03:50:28 diagd 2.1.0 [P23TAEW] INFO: EnvoyConfig: Generating V3

 2022-01-24 03:50:28 diagd 2.1.0 [P23TAEW] WARNING: No active listeners at all; check your Listener and Host configuration

 2022-01-24 03:50:28 diagd 2.1.0 [P23TAEW] INFO: configuration updated (incremental) from snapshot snapshot (S48 L0 G3 C1)

 time="2022-01-24 03:50:28.5671" level=info msg="Loaded file /ambassador/envoy/envoy.json" func=github.com/datawire/ambassador/v2/cmd/ambex.Decode file="/go/cmd/ambex/main.go:279" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:28.5674" level=info msg="Saved snapshot v14" func=github.com/datawire/ambassador/v2/cmd/ambex.csDump file="/go/cmd/ambex/main.go:369" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:28.5678" level=info msg="Pushing snapshot v14" func=github.com/datawire/ambassador/v2/cmd/ambex.updaterWithTicker file="/go/cmd/ambex/ratelimit.go:159" CMD=entrypoint PID=1 THREAD=/ambex

 2022-01-24 03:50:28 diagd 2.1.0 [P23TAEW] INFO: Updating Knative Ingress serving-srhc9-ksvc-c7jn5 status to ('ingress.networking.internal.knative.dev', 'default', {'observedGeneration': 1, 'conditions': [{'lastTransitionTime': '2022-01-24T03:50:28Z', 'status': 'True', 'type': 'LoadBalancerReady'}, {'lastTransitionTime': '2022-01-24T03:50:28Z', 'status': 'True', 'type': 'NetworkConfigured'}, {'lastTransitionTime': '2022-01-24T03:50:28Z', 'status': 'True', 'type': 'Ready'}], 'loadBalancer': {'ingress': [{'domainInternal': 'emissary-ingress.emissary.svc.cluster.local'}]}, 'privateLoadBalancer': {'ingress': [{'domainInternal': 'emissary-ingress.emissary.svc.cluster.local'}]}})

 2022-01-24 03:50:28 diagd 2.1.0 [P23TAEW] INFO: EnvoyConfig: Generating V3

 2022-01-24 03:50:28 diagd 2.1.0 [P23TAEW] WARNING: No active listeners at all; check your Listener and Host configuration

 2022-01-24 03:50:28 diagd 2.1.0 [P23TAEW] INFO: configuration updated (incremental) from snapshot snapshot (S48 L0 G7 C2)

 time="2022-01-24 03:50:28.7085" level=info msg="Loaded file /ambassador/envoy/envoy.json" func=github.com/datawire/ambassador/v2/cmd/ambex.Decode file="/go/cmd/ambex/main.go:279" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:28.7089" level=info msg="Saved snapshot v15" func=github.com/datawire/ambassador/v2/cmd/ambex.csDump file="/go/cmd/ambex/main.go:369" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:28.7094" level=info msg="Pushing snapshot v15" func=github.com/datawire/ambassador/v2/cmd/ambex.updaterWithTicker file="/go/cmd/ambex/ratelimit.go:159" CMD=entrypoint PID=1 THREAD=/ambex

 time="2022-01-24 03:50:38.5287" level=info msg="Memory Usage 0.16Gi (41%)" func="github.com/datawire/ambassador/v2/pkg/memory.(*MemoryUsage).Watch.func1" file="/go/pkg/memory/memory.go:39" CMD=entrypoint PID=1 THREAD=/memory

Is there anything else I forgot to do?

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior A clear and concise description of what you expected to happen.

Versions (please complete the following information):

  • Ambassador: [2.1.0]
  • Kubernetes environment [1.21.5]
  • Version [e.g. 1.8.1]

Additional context Add any other context about the problem here.

wanjunlei avatar Jan 24 '22 03:01 wanjunlei

the feild status. loadBalancer in ingresses.networking.internal.knative.dev has remove from knative 0.26 https://github.com/emissary-ingress/emissary/blob/master/python/ambassador/fetch/knative.py#L136 https://github.com/knative/networking/blob/release-0.25/pkg/apis/networking/v1alpha1/ingress_types.go#L303 https://github.com/knative/networking/blob/main/pkg/apis/networking/v1alpha1/ingress_types.go#L282

wei840222 avatar Jan 27 '22 06:01 wei840222

seems that knative has dropped emissary support https://github.com/knative/docs/pull/4670

alexanderGalushka avatar Feb 15 '22 02:02 alexanderGalushka

Do you have a Listener configured as you would need one in this case?

cindymullins-dw avatar Apr 01 '22 23:04 cindymullins-dw

Could someone enlighten me if knative support is supposed to work or not before I try it in vain?

According to these docs, things should work. But the documentation is a bit scarce.

aberres avatar Sep 16 '22 11:09 aberres

@aberres , we do have users who are using knative with Edge Stack and Emissary in our Slack. Our docs are admittedly light on this topic but you might try Emissary 2.x or later to see if it works for you. There are some older resources here and here and presentation. In Emissary 2.x you do need to configure Listeners overtly which is a change from the Emissary 1.x series.

cindymullins-dw avatar Dec 23 '22 04:12 cindymullins-dw