azure-functions-host icon indicating copy to clipboard operation
azure-functions-host copied to clipboard

Log stream not outputting anything in certain situations

Open guidojw opened this issue 10 months ago • 1 comments

I'm experiencing very odd behaviour with the log stream functionality. I implemented tests using az webapp log tail to keep checking the function app's output for a specific log in our tests. This works fine on our dev environment, but no logs seem to appear whatsoever when using this command on our staging function app. I get the usual "Now listening on ..." startup logs and a few other ones around that, but no request logs are outputted.

In the log stream tab of this function app on Azure Portal, I see the desired logs showing up when using the Application Insights setting. When using the Filesystem logs setting, I only see the same startup logs as earlier. I think this is related to the problem. Using Kudu I ssh'd into the OS and checked out the log files in /appsvctmp/volatile/logs/runtime/*.log, these show the same problem, only startup logs and nothing more. On our dev function app, both places show the desired other logs too.

I spent some time debugging and comparing all the function app settings, but have not found what is causing this so far.

Investigative information

The problem is not coupled to a specific invocation, but I attached them for debugging purposes. staging: no logs dev: working correctly

  • Timestamp: staging=2024-04-15 15:06:29.900 dev=2024-04-15 14:17:49.044
  • Function App version: ~4
  • Function App name: disclosed via Invocation ID
  • Function name(s) (as appropriate): all, testing on health_status
  • Invocation ID: staging=a2a4cb58-9d39-4716-a091-de7d12cce8c7 dev=8c1da834-a1ff-47d2-9161-7ed8b93be9af
  • Region: eu-west-1

Repro steps

  1. az webapp log tail --resource-group <resource_group_name> --name <function_app_name>

Expected behavior

See function execution logs.

Actual behavior

See only the start up logs.

Known workarounds

None, I don't think Application Insights has a log stream functionality.

Related information

Python 3.9,

host.json:

{
  "version": "2.0",
  "logging": {
    "fileLoggingMode": "always",
    "logLevel": {
      "Host.Results": "Error",
      "Function.health_status": "Debug",
      "Host.Aggregator": "Trace"
    },
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[3.3.0, 4.0.0)"
  }
}

Application Insights is coupled, ftp publishing credentials are off but scm is still on. Kudu dashboard is accessible.

guidojw avatar Apr 15 '24 21:04 guidojw

New information:

  • On the dev environment a diagnostic setting with category "FunctionAppLogs" was clicked together which was not configured in the IaC, this is an important difference with the other non-functional environments.
  • I tested out configuring "App Service logs", it seemed like this worked but this was only when the Log Stream was open somewhere recently.

Knowing this information, I tried out configuring App Service logs and adding a diagnostic setting pointing to a dummy storage account with a management policy deleting blobs after a day, this works like a charm.

The storage account solution however sounds a bit odd and like a non-solution to me. I'll keep this issue open to ask for confirmation if this is really the way to go.

guidojw avatar Apr 22 '24 20:04 guidojw