azure-functions-python-worker
azure-functions-python-worker copied to clipboard
Azure
Missing binaries to run numpy
Investigative information
Please provide the following:
- Function App version: v4
- Function App name: datadrift
- Region: eastus2
Repro steps
Provide the steps required to reproduce the problem:
Set numpy as a dependecy in the requirements.txt
Have the python function import numpy like import numpy as np
This is very similar to this issue but using the ubuntu vm as a builder didn't solve the issue
Expected behavior
To be able to import Numpy
Actual behavior
Fail to import numpy. This is the output of my durable function.
{
"createdTime": "2022-08-24T21:27:52Z",
"customStatus": null,
"input": "{\"hello\": \"world\"}",
"instanceId": "d9005b995aec46b6a15c66c8ec67fbbd",
"lastUpdatedTime": "2022-08-24T21:27:52Z",
"name": "orchestrator-function",
"output": "Orchestrator function 'orchestrator-function' failed: Activity function 'processing' failed: ImportError: Unable to import required dependencies:\nnumpy: \n\nIMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!\n\nImporting the numpy C-extensions failed. This error can happen for\nmany r
easons, often due to issues with your setup or how NumPy was\ninstalled.\n\nWe have compiled some common reasons and troubleshooting tips at:\n\n https://numpy.org/devdocs/user/troubleshooting-importerror.html\n\nPlease note and check the following:\n\n * The Python version is: Python3.8 from \"/usr/local/bin/pyt
hon\"\n * The NumPy version is: \"1.23.2\"\n\nand make sure that they are the versions you expect.\nPlease carefully study the documentation linked above for further help.\n\nOriginal error was: No module named 'numpy.core._multiarray_umath'\n. Please check the requirements.txt file for the missing module. For more
info, please refer the troubleshooting guide: https://aka.ms/functions-modulenotfound \n {\"$type\":\"System.Exception, System.Private.CoreLib\",\"ClassName\":\"System.Exception\",\"Message\":\" ImportError: Unable to import required dependencies:\\nnumpy: \\n\\nIMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE
THIS ISSUE!\\n\\nImporting the numpy C-extensions failed. This error can happen for\\nmany reasons, often due to issues with your setup or how NumPy was\\ninstalled.\\n\\nWe have compiled some common reasons and troubleshooting tips at:\\n\\n https://numpy.org/devdocs/user/troubleshooting-importerror.html\\n\\nPle
ase note and check the following:\\n\\n * The Python version is: Python3.8 from \\\"/usr/local/bin/python\\\"\\n * The NumPy version is: \\\"1.23.2\\\"\\n\\nand make sure that they are the versions you expect.\\nPlease carefully study the documentation linked above for further help.\\n\\nOriginal error was: No modu
le named 'numpy.core._multiarray_umath'\\n. Please check the requirements.txt file for the missing module. For more info, please refer the troubleshooting guide: https://aka.ms/functions-modulenotfound\",\"Data\":null,\"InnerException\":{\"$type\":\"Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException, Microsoft.A
zure.WebJobs.Script\",\"IsUserException\":false,\"RemoteStackTrace\":\" File \\\"/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/dispatcher.py\\\", line 355, in _handle__function_load_request\\n func = loader.load_function(\\n File \\\"/azure-functions-host/workers/python/3.8/LINUX/X64/
azure_functions_worker/utils/wrappers.py\\\", line 42, in call\\n raise extend_exception_message(e, message)\\n File \\\"/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/utils/wrappers.py\\\", line 40, in call\\n return func(*args, **kwargs)\\n File \\\"/azure-functions-host/workers/p
ython/3.8/LINUX/X64/azure_functions_worker/loader.py\\\", line 127, in load_function\\n mod = importlib.import_module(fullmodname)\\n File \\\"/usr/local/lib/python3.8/importlib/__init__.py\\\", line 127, in import_module\\n return _bootstrap._gcd_import(name[level:], package, level)\\n File \\\"<frozen impor
tlib._bootstrap>\\\", line 1014, in _gcd_import\\n File \\\"<frozen importlib._bootstrap>\\\", line 991, in _find_and_load\\n File \\\"<frozen importlib._bootstrap>\\\", line 975, in _find_and_load_unlocked\\n File \\\"<frozen importlib._bootstrap>\\\", line 671, in _load_unlocked\\n File \\\"<frozen importlib._b
ootstrap_external>\\\", line 843, in exec_module\\n File \\\"<frozen importlib._bootstrap>\\\", line 219, in _call_with_frames_removed\\n File \\\"/home/site/wwwroot/processing/__init__.py\\\", line 6, in <module>\\n from src.datadrift import report_data_drift\\n File \\\"/home/site/wwwroot/src/datadrift.py\\\"
, line 1, in <module>\\n import pandas as pd\\n File \\\"/home/site/wwwroot/.python_packages/lib/site-packages/pandas/__init__.py\\\", line 16, in <module>\\n raise ImportError(\\n\",\"RemoteMessage\":\"ImportError: Unable to import required dependencies:\\nnumpy: \\n\\nIMPORTANT: PLEASE READ THIS FOR ADVICE O
N HOW TO SOLVE THIS ISSUE!\\n\\nImporting the numpy C-extensions failed. This error can happen for\\nmany reasons, often due to issues with your setup or how NumPy was\\ninstalled.\\n\\nWe have compiled some common reasons and troubleshooting tips at:\\n\\n https://numpy.org/devdocs/user/troubleshooting-importerro
r.html\\n\\nPlease note and check the following:\\n\\n * The Python version is: Python3.8 from \\\"/usr/local/bin/python\\\"\\n * The NumPy version is: \\\"1.23.2\\\"\\n\\nand make sure that they are the versions you expect.\\nPlease carefully study the documentation linked above for further help.\\n\\nOriginal err
or was: No module named 'numpy.core._multiarray_umath'\\n. Please check the requirements.txt file for the missing module. For more info, please refer the troubleshooting guide: https://aka.ms/functions-modulenotfound\",\"RemoteTypeName\":null,\"Message\":\"Result: Failure\\nException: ImportError: Unable to import re
quired dependencies:\\nnumpy: \\n\\nIMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!\\n\\nImporting the numpy C-extensions failed. This error can happen for\\nmany reasons, often due to issues with your setup or how NumPy was\\ninstalled.\\n\\nWe have compiled some common reasons and troubleshooting
tips at:\\n\\n https://numpy.org/devdocs/user/troubleshooting-importerror.html\\n\\nPlease note and check the following:\\n\\n * The Python version is: Python3.8 from \\\"/usr/local/bin/python\\\"\\n * The NumPy version is: \\\"1.23.2\\\"\\n\\nand make sure that they are the versions you expect.\\nPlease carefu
lly study the documentation linked above for further help.\\n\\nOriginal error was: No module named 'numpy.core._multiarray_umath'\\n. Please check the requirements.txt file for the missing module. For more info, please refer the troubleshooting guide: https://aka.ms/functions-modulenotfound\\nStack: File \\\"/azur
e-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/dispatcher.py\\\", line 355, in _handle__function_load_request\\n func = loader.load_function(\\n File \\\"/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/utils/wrappers.py\\\", line 42, in call\\n raise extend_excep
tion_message(e, message)\\n File \\\"/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/utils/wrappers.py\\\", line 40, in call\\n return func(*args, **kwargs)\\n File \\\"/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/loader.py\\\", line 127, in load_function\\n
mod = importlib.import_module(fullmodname)\\n File \\\"/usr/local/lib/python3.8/importlib/__init__.py\\\", line 127, in import_module\\n return _bootstrap._gcd_import(name[level:], package, level)\\n File \\\"<frozen importlib._bootstrap>\\\", line 1014, in _gcd_import\\n File \\\"<frozen importlib._bootstr
ap>\\\", line 991, in _find_and_load\\n File \\\"<frozen importlib._bootstrap>\\\", line 975, in _find_and_load_unlocked\\n File \\\"<frozen importlib._bootstrap>\\\", line 671, in _load_unlocked\\n File \\\"<frozen importlib._bootstrap_external>\\\", line 843, in exec_module\\n File \\\"<frozen importlib._bootst
rap>\\\", line 219, in _call_with_frames_removed\\n File \\\"/home/site/wwwroot/processing/__init__.py\\\", line 6, in <module>\\n from src.datadrift import report_data_drift\\n File \\\"/home/site/wwwroot/src/datadrift.py\\\", line 1, in <module>\\n import pandas as pd\\n File \\\"/home/site/wwwroot/.python
_packages/lib/site-packages/pandas/__init__.py\\\", line 16, in <module>\\n raise ImportError(\\n\",\"Data\":{\"$type\":\"System.Collections.ListDictionaryInternal, System.Private.CoreLib\"},\"InnerException\":null,\"HelpLink\":null,\"Source\":\"System.Private.CoreLib\",\"HResult\":-2146233088,\"StackTrace\":\"
at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in /src/azure-functions-host/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:line 96\\n at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Obje
ct[] parameters) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionInvokerBase.cs:line 82\\n at Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator.Coerce[T](Task`1 src) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionGenerator.cs:line 225\\n at Microsoft.Azure.WebJo
bs.Host.Executors.FunctionInvoker`2.InvokeAsync(Object instance, Object[] arguments) in C:\\\\projects\\\\azure-webjobs-sdk-rqm4t\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionInvoker.cs:line 52\\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker in
voker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in C:\\\\projects\\\\azure-webjobs-sdk-rqm4t\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\F
unctionExecutor.cs:line 581\\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance, ParameterHelper parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in C:\\\\projects\\\\azure-webjobs-sdk-rqm4t\\\\src\\\\Microsoft.
Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 527\\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, Cancellation
Token cancellationToken) in C:\\\\projects\\\\azure-webjobs-sdk-rqm4t\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 306\\n at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in /src/azure-functions-
host/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:line 96\\n at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionInvokerBase.cs:line 82\\n at Microsoft.Azure.WebJobs.Script.Description.
FunctionGenerator.Coerce[T](Task`1 src) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionGenerator.cs:line 225\\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker`2.InvokeAsync(Object instance, Object[] arguments) in C:\\\\projects\\\\azure-webjobs-sdk-rqm4t\\\\src\\\\Microsoft.Azure.Web
Jobs.Host\\\\Executors\\\\FunctionInvoker.cs:line 52\\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTim
eout, TimeSpan timerInterval, IFunctionInstance instance) in C:\\\\projects\\\\azure-webjobs-sdk-rqm4t\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 581\\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance, ParameterHel
per parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in C:\\\\projects\\\\azure-webjobs-sdk-rqm4t\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 527\\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFuncti
onInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in C:\\\\projects\\\\azure-webjobs-sdk-rqm4t\\\\src\\\\Microsoft.Azure.WebJobs.Host\\\\Executors\\\\FunctionExecutor.cs:line 306\"},\"
HelpURL\":null,\"StackTraceString\":null,\"RemoteStackTraceString\":null,\"RemoteStackIndex\":0,\"ExceptionMethod\":null,\"HResult\":-2146233088,\"Source\":null,\"WatsonBuckets\":null}",
"runtimeStatus": "Failed"
}
Known workarounds
reinstall numpy on the host. Pip should make sure the c libraries are presents. Thing is I don't have access to the host as its serverless
Related information
Provide any related information
-
Programming language used : Python
-
Source: I am using this async example but with numpy as a depedency for math processing.
Source
This fails on this code:# processing.py
import numpy as np
In other word is there any workaround to make sure that the binary necessary for numpy are present on the host?
I found a fix for my issue.
In the azure-pipelines.yaml building my Azure function zip, I now run this command:
pip install --only-binary=:all: --target="./.python_packages/lib/site-packages" -r ./requirements.txt
Specifically asking for --only-binary
This solved my issue of missing binaries once the function is running
