jx icon indicating copy to clipboard operation
jx copied to clipboard

JX3 Logs are not getting stored under GCP storage

Open tgelpi-bot opened this issue 4 years ago • 4 comments

Summary

JX3/GKE/GSM environment does not store logs under GCP storage even when logging is enabled.

Steps to reproduce the behavior

  1. Build an OOTB JX3/GKE/GSM using Terraform

  2. Validate that logging is enabled with Octant under config map terraform-jx-requirements (default namespace)

  3. Validate log entries under jx-requirements.yml

  4. Validate the pod jx-build-controller logs and entry: long term storage for logs is being used, bucket gs://logs-jXXXX

  5. Expected behavior

Log files should reside under GCP buckets

Actual behavior

Log files do not reside under GCP buckets

Workaround steps to resolve

1, Delete the current pod jx-build-controller (another pod should automatically start) 2. Check the log of the new jx-build-contoller pod to see the meessage: "long term storage for logs is being used, bucket gs://logs-XXXXX" 3. Check the GCP storage under porta.

Jx version

The output of jx version is:

version: 3.1.76

Diagnostic information

The output of jx diagnose version is:

COPY OUTPUT HERE

Kubernetes cluster

Kubectl version

The output of kubectl version --client is:

Copy output here

Operating system / Environment

tgelpi-bot avatar Dec 26 '20 14:12 tgelpi-bot

This problem appears to be intermittent. I find that sometimes when creating a new environment the logs are configured. Most times I have to restart the jx-build-controller pod to set long term storage.

tgelpi-bot avatar Jan 04 '21 15:01 tgelpi-bot

Not sure if this issue was acknowledged. Each time a I build a new environment, I need to restart the jx-build-controller pod to set long term storage.

tgelpi-bot avatar Feb 05 '21 17:02 tgelpi-bot

Hey @tgelpi-bot , is this still an issue you are seeing?

tomhobson avatar Mar 12 '22 14:03 tomhobson

Yes this issue is still outstanding.

I just built a new GKE environment recently and checking jx-build-controller pod logs I see:

{"timestamp":"2022-03-19T15:33:01.73282139Z","message":"long term storage for logs is not configured in cluster requirements","severity":"INFO","context":{}}

After deleting the pod when it restarts I now see in the logs:

{"timestamp":"2022-03-19T19:36:45.33661076Z","message":"long term storage for logs is being used, bucket gs://logs-jx3ggg-490a8a050b54","severity":"INFO","context":{}}

tgelpi-bot avatar Mar 19 '22 19:03 tgelpi-bot