azure-functions-python-worker icon indicating copy to clipboard operation
azure-functions-python-worker copied to clipboard

[BUG] Init Indexing leads to exception being not outputted on timeout

Open tspring418 opened this issue 1 year ago • 0 comments

Investigative information

Please provide the following:
  • Timestamp: started around may 10 20:43 UTC
  • Function App name: prd-modelo-preditivo(staging) hml-modelo-preditivo(staging)

Repro steps

Provide the steps required to reproduce the problem:

No repro at the moment, but basically since the push out of : PYTHON_ENABLE_INIT_INDEXING

We have seen an issue that has led to the exceptions during init leading to ten second timeout between host and language worker prevents logging of the exception causing the problem during this step. Which leads to confusion on why a failure may be occuring during startup of the language worker.

Indexing times using old indexing

 

percentile_AverageTime_30 | percentile_AverageTime_50 | percentile_AverageTime_75 | percentile_AverageTime_90 | percentile_AverageTime_95 | percentile_AverageTime_99 -- | -- | -- | -- | -- | -- 16.5 | 16.5 | 16.6 | 21.406 | 21.406 | 21.406

 

Expected behavior

Metadata call during Indexing should occur while if a timeout happens, it will return the exception that occurred during this time that leads to the timeout and log that exception as well.

Actual behavior

Only a timeout exception is shown System.TimeoutException : The operation has timed out. at Microsoft.Azure.WebJobs.Script.Grpc.GrpcWorkerChannel.PendingItem.OnTimeout() at /src/azure-functions-host/src/WebJobs.Script.Grpc/Channel/GrpcWorkerChannel.cs : 1764

While the captured exception isn't logged out during indexing.

Known workarounds

Provide a description of any known workarounds.

PYTHON_ENABLE_INIT_INDEXING = 0 in app settings will allow you to see the real exception of the failure

Contents of the requirements.txt file:

Provide the requirements.txt file to help us find out module related issues.

n/a

Related information

Provide any related information

(https://github.com/Azure/azure-functions-python-worker/blob/5bc5d39efffe86da25cacdf7d612a2bc03b32ec3/azure_functions_worker/constants.py#L68C32-L68C59)

ICM # 503543832

Source
-->

tspring418 avatar May 15 '24 00:05 tspring418