Enmasse IoT http Adapter seems to continuously crash

Open astoycos opened this issue 5 years ago • 0 comments

Describe the bug A clear and concise description of what the bug is.

Trying to start Enmasse IoT but when I try to curl POST to the http adapter I get the following error

content-length: 92
cache-control: no-cache
content-type: text/html
connection: close

<html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>

And then when I look in the logs of the http adapter for the adapter log stream I see the following before it goes into crash loop backoff:

Hono HTTP Adapter (v1.0.3)
using Spring Boot (v2.1.5.RELEASE)

Go to https://enmasse.io for more information.

2020-06-01T17:38:07.814Z INFO  [Application] Starting Application v1.0.3 on iot-http-adapter-5d8ff8cfbf-j2zjh with PID 1 (/iot-http-adapter.jar started by 1000570000 in /)
2020-06-01T17:38:07.818Z INFO  [Application] No active profile set, falling back to default profiles: default
2020-06-01T17:38:19.319Z INFO  [Application] Started Application in 15.607 seconds (JVM running for 18.397)
2020-06-01T17:38:19.321Z INFO  [Application$$EnhancerBySpringCGLIB$$68757685] running on Java VM [version: 11.0.6, name: OpenJDK 64-Bit Server VM, vendor: Oracle Corporation, max memory: 61MB, processors: 1]
2020-06-01T17:38:19.510Z INFO  [VertxBasedHttpProtocolAdapter] using OpenTracing Tracer implementation [io.opentracing.noop.NoopTracerImpl]
2020-06-01T17:38:19.608Z INFO  [HonoConnectionImpl] using OpenTracing implementation [io.opentracing.noop.NoopTracerImpl]
2020-06-01T17:38:19.619Z INFO  [HonoConnectionImpl] using OpenTracing implementation [io.opentracing.noop.NoopTracerImpl]
2020-06-01T17:38:19.706Z INFO  [HonoConnectionImpl] using OpenTracing implementation [io.opentracing.noop.NoopTracerImpl]
2020-06-01T17:38:19.717Z INFO  [HonoConnectionImpl] using OpenTracing implementation [io.opentracing.noop.NoopTracerImpl]
2020-06-01T17:38:19.722Z INFO  [HonoConnectionImpl] using OpenTracing implementation [io.opentracing.noop.NoopTracerImpl]
2020-06-01T17:38:19.726Z INFO  [HonoConnectionImpl] using OpenTracing implementation [io.opentracing.noop.NoopTracerImpl]
2020-06-01T17:38:19.804Z INFO  [HonoConnectionImpl] using OpenTracing implementation [io.opentracing.noop.NoopTracerImpl]
2020-06-01T17:38:19.805Z INFO  [HonoConnectionImpl] using OpenTracing implementation [io.opentracing.noop.NoopTracerImpl]
2020-06-01T17:38:22.105Z INFO  [VertxBasedHttpProtocolAdapter] Vertx native support: false
2020-06-01T17:38:22.114Z INFO  [VertxBasedHttpProtocolAdapter] limiting size of inbound request body to 2048 bytes
2020-06-01T17:38:22.128Z INFO  [SpringBasedHonoPasswordEncoder] using BCrypt [strength: 10] with PRNG [NativePRNGNonBlocking] for encoding clear text passwords
2020-06-01T17:38:22.215Z INFO  [VertxBasedHttpProtocolAdapter] using OpenSSL [version: OpenSSL 1.0.1e-fips 11 Feb 2013] instead of JDK's default SSL engine
2020-06-01T17:38:22.216Z INFO  [VertxBasedHttpProtocolAdapter] enabling secure protocol [TLSv1.2]
2020-06-01T17:38:22.218Z INFO  [VertxBasedHttpProtocolAdapter] enabling client authentication using certificates [org.eclipse.hono.service.auth.ValidityBasedTrustOptions]
2020-06-01T17:38:22.708Z WARN  [BlockedThreadChecker] Thread Thread[vert.x-eventloop-thread-0,5,main]=Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 2800 ms, time limit is 2000 ms
2020-06-01T17:38:24.318Z INFO  [VertxBasedHttpProtocolAdapter] secure http server listening on 0.0.0.0:8443
2020-06-01T17:38:24.320Z INFO  [VertxBasedHealthCheckServer] registering additional resource: Prometheus Registry Scraper [endpoint: /prometheus]
2020-06-01T17:38:24.321Z INFO  [VertxBasedHealthCheckServer] won't start secure health checks HTTP server: no bind address configured.
2020-06-01T17:38:24.326Z INFO  [VertxBasedHealthCheckServer] successfully started insecure health checks HTTP server
2020-06-01T17:38:24.326Z INFO  [VertxBasedHealthCheckServer] readiness probe available at http://0.0.0.0:8088/readiness
2020-06-01T17:38:24.327Z INFO  [VertxBasedHealthCheckServer] liveness probe available at http://0.0.0.0:8088/liveness
2020-06-01T17:38:24.416Z INFO  [VertxBasedHttpProtocolAdapter] connected to AMQP Messaging Network
2020-06-01T17:38:25.184Z INFO  [VertxBasedHttpProtocolAdapter] connected to Credentials service
2020-06-01T17:38:25.188Z INFO  [VertxBasedHttpProtocolAdapter] connected to Tenant service
2020-06-01T17:38:25.190Z INFO  [VertxBasedHttpProtocolAdapter] connected to Device Connection service
2020-06-01T17:38:25.194Z INFO  [VertxBasedHttpProtocolAdapter] connected to Command & Control
2020-06-01T17:38:25.194Z INFO  [VertxBasedHttpProtocolAdapter] connected to Device Registration service
2020-06-01T17:41:41.223Z INFO  [AbstractRequestResponseClient] enabling caching of responses from tenant
2020-06-01T17:41:42.314Z INFO  [AbstractRequestResponseClient] enabling caching of responses from credentials

And for the qdr-cfg log stream I see

E0601 17:37:50.618192       1 reflector.go:280] pkg/client/informers/externalversions/factory.go:109: Failed to watch *v1alpha1.IoTProject: unknown (get iotprojects.iot.enmasse.io)
{"level":"info","ts":1591033071.6218357,"logger":"cmd","msg":"Should queue myapp/iot -> false"}

To Reproduce Steps to reproduce the behavior:

Go to the Enmasse IoT and follow the starting instructions

Expected behavior

The http adapter to work, and for the curl POST to be received by the hono client

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem her

Running on an Openshift 4.4.5 Cluster on AWS

Jun 01 '20 17:06 astoycos