⭐ [Enhancement]: Introduce Logging Sink & Settings

[JerryNixon]

What is it

• Add global filters
• Add a file sink

Log sink details

Current syntax: docs

{
  "runtime": {
    ...
    "telemetry": {
      "application-insights": {
        "enabled": true,
        "connection-string": "@env('app-insights-connection-string')"
      }
    },
    ...
  }
}

Sample future configuration

{
  "runtime": {
    ...
    "telemetry": {
       "log-level": { // log filters are only at the global level
          "Default": "Warning",
          "Microsoft": "Error",
          "Microsoft.AspNetCore": "Information"
       },
      "application-insights": {
        "enabled": true,
        "connection-string": "@env('app-insights-connection-string')"
      },
      "file": {
        "enabled": true,
        "path": "/logs/dab-log.txt",
        "rollingInterval": "Day",
        "retainedFileCountLimit": 7,
        "fileSizeLimitBytes": 10485760
      }
    }
    ...
  }
}

Initial discussions:

Global log-level

The global log-level, which is similar to how global cache TTL works, can be inherited by a sink.

Move Global log-level

The current (unadvertised) global log-level will be moved from under /runtime to under /runtime/telemetry.

File Explanation:

• File Logging: The file sink is enabled to store logs locally in the path /logs/dab-log.txt. It includes additional configurations for file management:
  • Rolling Interval: Specifies how often the logs are rolled over to a new file, in this case, daily ("rollingInterval": "Day").

/logs/dab-log-20240923.txt
/logs/dab-log-20240924.txt

• Retained File Limit: Limits the number of files stored to prevent excessive disk space usage, here set to retain up to 7 log files ("retainedFileCountLimit": 7).
• File Size Limit: Prevents any log file from exceeding 10 MB ("fileSizeLimitBytes": 10485760).

/logs/dab-log-20240923.txt
/logs/dab-log-20240923_001.txt
/logs/dab-log-20240923_002.txt

Container Files:

The developer is required to mount a volume for File sink.

docker run -v /my/host/logs:/logs my-container

The developer may also bind a mount for File sinl.

docker run --mount type=bind,source=/my/host/logs,target=/logs my-container

Related issues to potentially close

• #2252
• #1818
• #1725

[JerryNixon]

Related https://github.com/Azure/data-api-builder/issues/1818