internal server errors found on remote web ui for otel-demo app

[sophiaxu0424]

Bug Report

Which version of the demo you are using? (please provide either a specific commit hash

Symptom

opentelemetry-demo is seeing internal server errors in in a remote machine (otel-app was deployed on a redhat VM) web browser:

And clicking the buttons for go shopping or adding product into cart doesn't work normally for now.

What is the expected behavior? Web UI should be working normal as a shopping app, and no internal server error (500) in the console.

What do you expect to see? Web UI should be working normal as a shopping app, and no internal server error (500) in the console.

What is the actual behavior? shopping button doesn't work, console errors

Please describe the actual behavior experienced.

Reproduce

Could you provide the minimum required steps to resolve the issue you're seeing?

follow the steps showing on docker deployment for otel-demo app

We will close this issue if:

• The steps you provided are complex.
• If we can not reproduce the behavior you're reporting.

Additional Context

Please feel free to add any other context about the problem here.

[sophiaxu0424]

Hi @morningspace FYI.

[julianocosta89]

hello @sophiaxu0424 thanks for reporting this!

Any chance you are changing the collector config? If yes, would you be able to share the final config in this thread?

None of those errors should be there, but the CORS missing allow origin is explicitly configured on the collector, so I'm wondering what is different on your deployment.

[morningspace]

Thanks @julianocosta89 for the quick reply.

Just to add, we are also seeing that the frontend service keeps reporting errors as below.

PageNotFoundError: Cannot find module for page: /500
    at getPagePath (/app/node_modules/next/dist/server/require.js:94:15)
    at requirePage (/app/node_modules/next/dist/server/require.js:99:22)
    at /app/node_modules/next/dist/server/load-components.js:98:84
    at ZoneDelegate.invoke (/app/node_modules/zone.js/bundles/zone.umd.js:400:30)
    at Zone.run (/app/node_modules/zone.js/bundles/zone.umd.js:160:47)
    at /app/node_modules/zone.js/bundles/zone.umd.js:1318:38
    at ZoneDelegate.invokeTask (/app/node_modules/zone.js/bundles/zone.umd.js:434:35)
    at Zone.runTask (/app/node_modules/zone.js/bundles/zone.umd.js:205:51)
    at drainMicroTaskQueue (/app/node_modules/zone.js/bundles/zone.umd.js:620:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
 ⨯ Error: Could not parse target name ""
    at new InternalChannel (/app/node_modules/@grpc/grpc-js/build/src/internal-channel.js:139:19)
    at new ChannelImplementation (/app/node_modules/@grpc/grpc-js/build/src/channel.js:35:32)
    at new Client (/app/node_modules/@grpc/grpc-js/build/src/client.js:66:36)
    at new ServiceClientImpl (/app/node_modules/@grpc/grpc-js/build/src/make-client.js:58:5)
    at /app/.next/server/pages/api/products/[productId].js:1:1399
Unhandled Promise rejection: Cannot find module for page: /500 ; Zone: <root> ; Task: Promise.then ; Value: PageNotFoundError: Cannot find module for page: /500
    at getPagePath (/app/node_modules/next/dist/server/require.js:94:15)
    at requirePage (/app/node_modules/next/dist/server/require.js:99:22)
    at /app/node_modules/next/dist/server/load-components.js:98:84
    at ZoneDelegate.invoke (/app/node_modules/zone.js/bundles/zone.umd.js:400:30)
    at Zone.run (/app/node_modules/zone.js/bundles/zone.umd.js:160:47)
    at /app/node_modules/zone.js/bundles/zone.umd.js:1318:38
    at ZoneDelegate.invokeTask (/app/node_modules/zone.js/bundles/zone.umd.js:434:35)
    at Zone.runTask (/app/node_modules/zone.js/bundles/zone.umd.js:205:51)
    at drainMicroTaskQueue (/app/node_modules/zone.js/bundles/zone.umd.js:620:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ENOENT'
} PageNotFoundError: Cannot find module for page: /500
    at getPagePath (/app/node_modules/next/dist/server/require.js:94:15)
    at requirePage (/app/node_modules/next/dist/server/require.js:99:22)
    at /app/node_modules/next/dist/server/load-components.js:98:84
    at ZoneDelegate.invoke (/app/node_modules/zone.js/bundles/zone.umd.js:400:30)
    at Zone.run (/app/node_modules/zone.js/bundles/zone.umd.js:160:47)
    at /app/node_modules/zone.js/bundles/zone.umd.js:1318:38
    at ZoneDelegate.invokeTask (/app/node_modules/zone.js/bundles/zone.umd.js:434:35)
    at Zone.runTask (/app/node_modules/zone.js/bundles/zone.umd.js:205:51)
    at drainMicroTaskQueue (/app/node_modules/zone.js/bundles/zone.umd.js:620:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

[sophiaxu0424]

hello @sophiaxu0424 thanks for reporting this!

Any chance you are changing the collector config? If yes, would you be able to share the final config in this thread?

None of those errors should be there, but the CORS missing allow origin is explicitly configured on the collector, so I'm wondering what is different on your deployment.

Hi Julian,

exporters:
  otlp/instana:
    endpoint: http://<my_endpoint_ip_of_this_server>:4317
    tls:
      insecure: true
processors:
  resource:
    attributes:
      - key: service.instance.id
        value: sophiademo-dev
        action: upsert
service:
  pipelines:
    traces:
      processors: [transform, resource, batch]
      exporters: [otlp/instana, debug, spanmetrics]
    metrics:
      receivers: [httpcheck/frontendproxy, otlp, prometheus, redis, spanmetrics]
      processors: [resource, batch]
      exporters: [otlp/instana, debug]

[emilyashley]

I am also running into this issue.

[puckpuck]

This looks to be a CORS issue. The Demo's default collector configuration solves for this. You need to ensure the collector used to receive telemetry from a web browser is configured in a similar fashion.

[morningspace]

Hi @puckpuck , thanks for your looking into this issue.

I think we did not change the receiver configuration. What @sophiaxu0424 pasted above is the customized pieces that we added by following the make procedure, i.e.: those contents in otelcol-config-extras.yml which will be merged into the default configuration when running make start. And, we only touched exporters, processors, service.

[morningspace]

One quick update: I noticed there's a setting in .env:

PUBLIC_OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:8080/otlp-http/v1/traces

This may need to be updated to reflect a remote host address instead of the default localhost. After making this change, and restart the demo app, I can see the CORS issue is missing. But still seeing the 500 Internal Server Error. This probably maps to the error I'm seeing here.

[sophiaxu0424]

This looks to be a CORS issue. The Demo's default collector configuration solves for this. You need to ensure the collector used to receive telemetry from a web browser is configured in a similar fashion.

Hi @puckpuck ,I did not touch any changes here in this yaml file: otelcol-config.yml, and only changed the config in otelcol-config-extras.yml, it reported the above internal server error.

I've reverted all of my changes in this repo, to use the default config in otelcol-config.yml and otelcol-config-extras.yml, I am still seeing the same error (500 internal server error).

FYI @morningspace . Thanks.

[puckpuck]

Can you share container logs for the frontend and currency services?

[sophiaxu0424]

Hi @puckpuck logs for frontend from my side

 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/data.js:1:1259)
    at o.a (.next/server/webpack-api-runtime.js:1:866)
    at 9864 (.next/server/pages/api/data.js:1:1128)
    at o (.next/server/webpack-api-runtime.js:1:127)
    at <unknown> (.next/server/pages/api/data.js:1:838)
    at o.a (.next/server/webpack-api-runtime.js:1:866)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/cart.js:1:2278)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/cart.js:1:2278)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/cart.js:1:2278)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/checkout.js:1:2681)
    at o.a (.next/server/webpack-api-runtime.js:1:866)
    at 6815 (.next/server/pages/api/checkout.js:1:2544)
    at o (.next/server/webpack-api-runtime.js:1:127)
    at <unknown> (.next/server/pages/api/checkout.js:1:2005)
    at o.a (.next/server/webpack-api-runtime.js:1:866)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/recommendations.js:1:220)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/cart.js:1:2278)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/cart.js:1:2278)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/cart.js:1:2278)
 ⨯ Error: Could not parse target name ""
    at <unknown> (.next/server/pages/api/products/[productId].js:1:1037)
``

[sophiaxu0424]

and about logs for currency

[Error] File: /opentelemetry-cpp/exporters/otlp/src/otlp_grpc_log_record_exporter.cc:173 [OTLP LOG GRPC Exporter] Export() failed: failed to connect to all addresses; last error: UNKNOWN: ipv4:172.18.0.9:4317: Failed to connect to remote host: Connection refused
[Error] File: /opentelemetry-cpp/exporters/otlp/src/otlp_grpc_log_record_exporter.cc:173 [OTLP LOG GRPC Exporter] Export() failed: failed to connect to all addresses; last error: UNKNOWN: ipv4:172.18.0.9:4317: Failed to connect to remote host: Connection refused

[sophiaxu0424]

I've checked my other opentelemetry-demo environment, currency logs is empty, and frontend see below

PageNotFoundError: Cannot find module for page: /500
    at getPagePath (/app/node_modules/next/dist/server/require.js:94:15)
    at requirePage (/app/node_modules/next/dist/server/require.js:99:22)
    at /app/node_modules/next/dist/server/load-components.js:98:84
    at ZoneDelegate.invoke (/app/node_modules/zone.js/bundles/zone.umd.js:400:30)
    at Zone.run (/app/node_modules/zone.js/bundles/zone.umd.js:160:47)
    at /app/node_modules/zone.js/bundles/zone.umd.js:1318:38
    at ZoneDelegate.invokeTask (/app/node_modules/zone.js/bundles/zone.umd.js:434:35)
    at Zone.runTask (/app/node_modules/zone.js/bundles/zone.umd.js:205:51)
    at drainMicroTaskQueue (/app/node_modules/zone.js/bundles/zone.umd.js:620:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
 ⨯ Error: Could not parse target name ""
    at new InternalChannel (/app/node_modules/@grpc/grpc-js/build/src/internal-channel.js:139:19)
    at new ChannelImplementation (/app/node_modules/@grpc/grpc-js/build/src/channel.js:35:32)
    at new Client (/app/node_modules/@grpc/grpc-js/build/src/client.js:66:36)
    at new ServiceClientImpl (/app/node_modules/@grpc/grpc-js/build/src/make-client.js:58:5)
    at /app/.next/server/pages/api/cart.js:1:940
    at o.a (/app/.next/server/webpack-api-runtime.js:1:866)
    at 1848 (/app/.next/server/pages/api/cart.js:1:799)
    at o (/app/.next/server/webpack-api-runtime.js:1:127)
    at /app/.next/server/pages/api/cart.js:1:2130
    at o.a (/app/.next/server/webpack-api-runtime.js:1:866)
Unhandled Promise rejection: Cannot find module for page: /500 ; Zone: <root> ; Task: Promise.then ; Value: PageNotFoundError: Cannot find module for page: /500
    at getPagePath (/app/node_modules/next/dist/server/require.js:94:15)
    at requirePage (/app/node_modules/next/dist/server/require.js:99:22)
    at /app/node_modules/next/dist/server/load-components.js:98:84
    at ZoneDelegate.invoke (/app/node_modules/zone.js/bundles/zone.umd.js:400:30)
    at Zone.run (/app/node_modules/zone.js/bundles/zone.umd.js:160:47)
    at /app/node_modules/zone.js/bundles/zone.umd.js:1318:38
    at ZoneDelegate.invokeTask (/app/node_modules/zone.js/bundles/zone.umd.js:434:35)
    at Zone.runTask (/app/node_modules/zone.js/bundles/zone.umd.js:205:51)
    at drainMicroTaskQueue (/app/node_modules/zone.js/bundles/zone.umd.js:620:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ENOENT'
} PageNotFoundError: Cannot find module for page: /500
    at getPagePath (/app/node_modules/next/dist/server/require.js:94:15)
    at requirePage (/app/node_modules/next/dist/server/require.js:99:22)
    at /app/node_modules/next/dist/server/load-components.js:98:84
    at ZoneDelegate.invoke (/app/node_modules/zone.js/bundles/zone.umd.js:400:30)
    at Zone.run (/app/node_modules/zone.js/bundles/zone.umd.js:160:47)
    at /app/node_modules/zone.js/bundles/zone.umd.js:1318:38
    at ZoneDelegate.invokeTask (/app/node_modules/zone.js/bundles/zone.umd.js:434:35)
    at Zone.runTask (/app/node_modules/zone.js/bundles/zone.umd.js:205:51)
    at drainMicroTaskQueue (/app/node_modules/zone.js/bundles/zone.umd.js:620:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

[morningspace]

I was having the same problem as @sophiaxu0424 does. And after I did some modifications, now I can see the pages from a remote machine. Here is what I did:

• The first thing I did is to change the value of PUBLIC_OTEL_EXPORTER_OTLP_TRACES_ENDPOINT in .env. It's default to http://localhost:8080/otlp-http/v1/traces which is definitely not true as I'm accessing remotely. This will avoid the browser keeps sending trace data to localhost, which I guess that's part of end-user monitoring.

PUBLIC_OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://<MY_REMOTE_HOST>:8080/otlp-http/v1/traces

• I also changed the value of ENV_PLATFORM from local to remote in .env, but it looks this is just an indicator for displaying purpose.

• The next thing I did is to add a new file src/frontend/pages/500.js. This will solve the Cannot find module for page: /500 errors as above. I think that's a bug. If it makes sense, I'd be happy to submit a PR for this.

export default function Custom500() {
  return <h1>500 - Server-side error occurred</h1>
}

[morningspace]

After I did the above changes, I can see the demo generally works. But still, I can see some errors when navigating through the pages. For example, I can see the following request is still looking for localhost:

http://localhost:8080//icons/CartIcon.svg?w=48&q=75

And, the following request showing 404.

http://<MY_REMOTE_HOST>:8080/images/products/undefined

[puckpuck]

Ahh, I think I missed that this was deployed to a RedHat VM and thus the local collector isn't available on the default name. Typically this is deployed to the local system where everything just works. It might make sense we add some docs about errors that may appear in the browser if not accessed on localhost.