Python Requests Heavy on Logging, Not Respecting Environment Log_Level Vars

[theblinkingusb]

Affected module Backend / Ingestion Framework

Describe the bug Webserver logging each request

To Reproduce Start a few ingestion pipelines and review the logs

Screenshots or steps to reproduce Examples: 192.xxx.xxx.xxx - - [12/Apr/2024:19:47:43 +0000] "GET /api/v1/system/config/jwks HTTP/1.1" 200 454 "-" "Java/17.0.10" 1 10.xxx.xxx.xxx - - [12/Apr/2024:19:47:43 +0000] "GET /api/v1/services/databaseServices/name/<redact_servicename> HTTP/1.1" 200 548 "-" "python-requests/2.31.0" 12 10.xxx.xxx.xxx - - [12/Apr/2024:19:47:43 +0000] "GET /api/v1/services/databaseServices/name/<redact_servicename> HTTP/1.1" 200 548 "-" "python-requests/2.31.0" 13 10.xxx.xxx.xxx - - [12/Apr/2024:19:47:43 +0000] "GET /api/v1/system/version HTTP/1.1" 200 99 "-" "python-requests/2.31.0" 0 10.xxx.xxx.xxx - - [12/Apr/2024:19:47:43 +0000] "GET /api/v1/system/version HTTP/1.1" 200 99 "-" "python-requests/2.31.0" 0 10.xxx.xxx.xxx - - [12/Apr/2024:19:47:43 +0000] "GET /api/v1/system/version HTTP/1.1" 200 99 "-" "python-requests/2.31.0" 1

/api/v1/tables/name/ , /api/v1/lineage routes also show

Expected behavior To respect environment logging settings or to be nullified - "I have tried adjusting openmetadata.config.logLevel to ERROR as well as setting LOG_LEVEL to ERROR in ExtraEnvs. I also noticed that some appenders were specified as TRACE in the image's openmetadata.yaml, so I also edited the k8s deployment to mount an adjusted openmetadata.yaml to mark everything as ERROR, but no luck."

Version:

• OpenMetadata version: Helm 1.3.2

Additional context Slack thread for reference - https://openmetadata.slack.com/archives/C02B6955S4S/p1712951450956009

[IceS2]

Hey @theblinkingusb, you can remove the server request logging by adding the following configuration to the server section in openmetadata.yaml

requestLog:
  appenders: [ ]

It should look like this:

server:
  rootPath: '/api/*'
  applicationConnectors:
    - type: http
      port: ${SERVER_PORT:-8585}
  adminConnectors:
    - type: http
      port: ${SERVER_ADMIN_PORT:-8586}
  requestLog:
    appenders: [ ]

We'll keep taking a look to find a better way to handle this but for now that should remove the excess logging

[theblinkingusb]

Thank you @IceS2!! yeah I was trying to do something like that but wasn't smart enough to figure it out :) I did your change and deployed 1.3.4 and went from 20k logs per hour down to 300, so that's awesome. Thanks again

[harshach]

@IceS2 anything to address here. If its docs lets update it and close it out