Monitoring feature is not working.

[artemnikitin2023]

trafficstars

Hello. I've successfully deployed Bridge v4.16 on vanilla k8s with oidc auth. All works fine except the monitoring feature, there is no Observe\Monitoring UI Tab at all and metrics at Compute - Nodes is not available (got 404 not found error). The monitoring plugin is deployed as a sidecar container in console pod, nginx listens localhost within a pod and the plugin is mapped via BRIDGE_PLUGINS="monitoring-plugin=http://localhost" environment variable. Also, i've set BRIDGE_PROMETHEUS_PUBLIC_URL, BRIDGE_ALERTMANAGER_PUBLIC_URL, BRIDGE_GRAFANA_PUBLIC_URL to the in cluster services (.svc.cluster.local), but looks like it's do nothing. There is no any error messages in logs so i don't understand why it's not working.

Is there anyone who have the similar setup and made it work? Please help. Regards.

[Vovanys]

@artemnikitin2023 мне помогло скачивание офф образа registry.redhat.io/openshift4/ose-monitoring-plugin-rhel9:v4.16.0-202408070337.p0.gaa339e4.assembly.stream.el9 после этого все заработало

[cloudcafetech]

@artemnikitin2023 Are you able to fix monitoring issue? Can you please share deployment ...

[Vovanys]

@cloudcafetech monitoring-plugin.zip Copy this static to webserver (nginx, apache) and set url in env deployment console BRIDGE_PLUGINS="monitoring-plugin=http://static-url/folder"

[cloudcafetech]

@Vovanys

Will try & let you know ... Tnx :)

[cloudcafetech]

@Vovanys

No luck :(

• Getting error console pod log

E1012 05:34:09.401603       1 handlers.go:173] GET request for "monitoring-plugin" plugin failed with 304 status code
E1012 05:34:09.401625       1 utils.go:41] Failed sending HTTP response body: http: request method or response status code does not allow body
E1012 05:34:10.649623       1 handlers.go:173] GET request for "monitoring-plugin" plugin failed with 304 status code
E1012 05:34:10.649651       1 utils.go:41] Failed sending HTTP response body: http: request method or response status code does not allow body
E1012 05:34:25.324392       1 handlers.go:173] GET request for "monitoring-plugin" plugin failed with 304 status code
E1012 05:34:25.324423       1 utils.go:41] Failed sending HTTP response body: http: request method or response status code does not allow body

• Plugins log

192.168.1.220 - - [12/Oct/2024:05:37:46 +0000] "GET /plugin-manifest.json HTTP/1.1" 304 -
192.168.1.220 - - [12/Oct/2024:05:38:46 +0000] "GET /plugin-manifest.json HTTP/1.1" 304 -
192.168.1.220 - - [12/Oct/2024:05:39:24 +0000] "GET /plugin-manifest.json HTTP/1.1" 304 -
192.168.1.220 - - [12/Oct/2024:05:39:25 +0000] "GET /plugin-manifest.json HTTP/1.1" 304 -
192.168.1.220 - - [12/Oct/2024:05:39:40 +0000] "GET /plugin-manifest.json HTTP/1.1" 304 -

If I use 4:16 version, Console POD not coming up (error Fatal glibc error: CPU does not support x86-64-v2) But

If I use below version, Console POD coming up d But there is no Observe\Monitoring UI Tab .

[artemnikitin2023]

I've added some env variables to console container, after this the Observe menu appears. Also you need monitoring plugin sidecar container. Here is a variables set i'm using:

      env:
      - name: BRIDGE_PLUGINS
        value: "monitoring-plugin=http://localhost:80"
      - name: BRIDGE_USER_AUTH
        value: "oidc"
      - name: BRIDGE_USER_AUTH_OIDC_CLIENT_ID
        value: "your oidc client id"
      - name: BRIDGE_USER_AUTH_OIDC_CLIENT_SECRET
        value: "your oidc client secret"
      - name: BRIDGE_USER_AUTH_OIDC_ISSUER_URL
        value: "oidc issuer url for example https://dex.example.com/"
      - name: BRIDGE_K8S_AUTH
        value: "oidc"
      - name: BRIDGE_USER_AUTH_OIDC_TOKEN_SCOPES
        value: "additional oidc token scopes for your environment"
      - name: BRIDGE_COOKIE_AUTHENTICATION_KEY_FILE
        value: "path to mounted generated secret"
      - name: BRIDGE_COOKIE_ENCRYPTION_KEY_FILE
        value: "path to mounted generated secret"
      - name: BRIDGE_USER_AUTH_OIDC_CA_FILE
        value: "path to mounted ca file in case of self signed certs"
      - name: BRIDGE_BASE_ADDRESS
        value: "https://console.example.com"
      - name: BRIDGE_V
        value: "log level"
      - name: BRIDGE_PROMETHEUS_PUBLIC_URL
        value: "https://prometheus.example.com"
      - name: BRIDGE_K8S_MODE_OFF_CLUSTER_THANOS
        value: "https://prometheus.example.com"
      - name: BRIDGE_THANOS_PUBLIC_URL
        value: "https://prometheus.example.com"
      - name: BRIDGE_ALERTMANAGER_PUBLIC_URL
        value: "https://alertmanager.example.com"
      - name: BRIDGE_GRAFANA_PUBLIC_URL
        value: "https://grafana.example.com"
      - name: BRIDGE_K8S_MODE_OFF_CLUSTER_ENDPOINT
        value: "https://kubernetes.default"
      - name: BRIDGE_K8S_MODE
        value: "off-cluster"
      - name: BRIDGE_K8S_MODE_OFF_CLUSTER_SKIP_VERIFY_TLS
        value: "true"

Also you will need to edit promQL queries to make all metrics work.

[cloudcafetech]

@artemnikitin2023 Great, By the way .. which OCP console version & what is the K8S version ?

[BlindB0]

Hi @artemnikitin2023! Did you solve the monitoring problem in vanila k8s? Can you share monitoring plugin sidecar container yaml code?

[gecube]

@artemnikitin2023 Hi ! I was able to get Observe tab on the left on open shift-console installed on top of vanilla cluster. Did you get any success?

[gecube]

@BlindB0

Can you share monitoring plugin sidecar container yaml code?

I changed the Dockerfile for 4.16 to this one:

FROM node:18.1.0-alpine AS builder

WORKDIR /usr/src/app

#RUN npm install --global yarn

ENV HUSKY=0

COPY package.json yarn.lock .
RUN yarn

COPY ./console-extensions.json ./tsconfig.json ./webpack.config.ts .
COPY ./locales ./locales
COPY ./src ./src
RUN yarn build

FROM amazonlinux:2023.6.20241121.0

RUN INSTALL_PKGS="nginx" && \
    dnf install -y --setopt=tsflags=nodocs $INSTALL_PKGS && \
    rpm -V $INSTALL_PKGS && \
    yum -y clean all --enablerepo='*' && \
    chown -R 1001:0 /var/lib/nginx /var/log/nginx /run && \
    chmod -R ug+rwX /var/lib/nginx /var/log/nginx /run

#USER 1001

COPY --from=builder /usr/src/app/dist /usr/share/nginx/html

ENTRYPOINT ["nginx", "-g", "daemon off;"]

and then deployed it to the cluster. Everything was relatively trivial.

[cognvn]

I was able to get Observe tab after install prometheuses.monitoring.coreos.com crd https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/main/example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml

[gecube]

@cognvn I am glad to hear it.

[cloudcafetech]

@gecube IT works for 4:16, looks like they moved from nginx to something different for newer version.

any updated Dockerfile for new version (4.18 +)?

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten /remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen. Mark the issue as fresh by commenting /remove-lifecycle rotten. Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci[bot]]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen. Mark the issue as fresh by commenting /remove-lifecycle rotten. Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.