BentoML
                                
                                 BentoML copied to clipboard
                                
                                    BentoML copied to clipboard
                            
                            
                            
                        bug: Exhaustive run during `bentoml build` on AzureDevOps
Describe the bug
cc https://bentoml.slack.com/archives/CKRANBHPH/p1658494302553029
TLDR: When running bentoml build locally, it works as expected. However, on AzureDevOps python agent, the process seems to hang.
To reproduce
Current progress:
- I seem to be able to reproduce the hanging locally, it seems like a bad combo of the bentoml CLI and our anti-malware... I am able to run locally by replacing the CLI commands (bentoml build) by a python script doing the same thing (bentoml.bentos.build_bentofile()), it behaves better... However, things are still stuck on the AzDO build agent... running python -m trace --trace myscript.py it seems stuff is happening, but I am still digesting it. In any case it seems there is no 'hanging' part so far (but I cannot let the script run forever, I already am at 170MB of trace for letting the bentoml.bentos.build_bentofile() for 5 minutes...)
Expected behavior
No response
Environment
bentoml: 1.0.0
python: 3.8.11
platform: Linux-3.10.0-1160.66.1.el7.x86_64-x86_64-with-glibc2.2.5
uid:gid: 0:0
pip_packages
aiofiles==0.8.0
aiohttp==3.8.1
aiosignal==1.2.0
anyio==3.6.1
appdirs==1.4.4
asgiref==3.5.2
async-timeout==4.0.2
attrs==21.4.0
bentoml==1.0.0
build==0.8.0
cattrs==22.1.0
certifi==2022.6.15
chardet==5.0.0
charset-normalizer==2.1.0
circus==0.17.1
click==8.1.3
cloudpickle==2.1.0
commonmark==0.9.1
conda==4.3.16
contextlib2==21.6.0
deepmerge==1.0.1
Deprecated==1.2.13
exceptiongroup==1.0.0rc8
filelock==3.7.1
frozenlist==1.3.0
fs==2.4.16
h11==0.13.0
idna==3.3
imageio==2.20.0
Jinja2==3.1.2
MarkupSafe==2.1.1
multidict==6.0.2
numpy==1.23.1
opencv-python==4.6.0.66
opentelemetry-api==1.9.0
opentelemetry-instrumentation==0.28b0
opentelemetry-instrumentation-aiohttp-client==0.28b0
opentelemetry-instrumentation-asgi==0.28b0
opentelemetry-sdk==1.9.0
opentelemetry-semantic-conventions==0.28b0
opentelemetry-util-http==0.28b0
packaging==21.3
pathspec==0.9.0
pep517==0.12.0
Pillow==9.2.0
pip-tools==6.8.0
prometheus-client==0.13.1
psutil==5.9.1
pycosat==0.6.3
Pygments==2.12.0
pynvml==11.4.1
pyparsing==3.0.9
python-dateutil==2.8.2
python-dotenv==0.20.0
python-json-logger==2.0.4
python-multipart==0.0.5
PyYAML==6.0
pyzmq==23.2.0
requests==2.28.1
rich==12.5.1
ruamel.yaml==0.17.21
ruamel.yaml.clib==0.2.6
schema==0.7.5
simple-di==0.1.5
six==1.16.0
sniffio==1.2.0
starlette==0.20.4
tomli==2.0.1
torch==1.12.0
torchvision==0.13.0
tornado==6.2
typing_extensions==4.3.0
urllib3==1.26.11
uvicorn==0.18.2
wrapt==1.14.1
yarl==1.7.2