Running ROMP in modular mode

[mkhoshle]

Hi,

I am making changes to the ROMP and I would like to run the code to see the results. I do python -m torch.utils.bottleneck romp.train --configs_yml='configs/v1_hrnet_3dpw_ft.yml'.

Traceback (most recent call last):
  File "/z/home/mahzad-khosh/env/romp2/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/z/home/mahzad-khosh/env/romp2/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/z/home/mahzad-khosh/env/romp2/lib/python3.8/site-packages/torch/utils/bottleneck/__main__.py", line 229, in <module>
    main()
  File "/z/home/mahzad-khosh/env/romp2/lib/python3.8/site-packages/torch/utils/bottleneck/__main__.py", line 192, in main
    with open(scriptfile, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: 'romp.train'

When I do python -m romp.train --configs_yml='configs/v1_hrnet_3dpw_ft.yml' it works fine. But adding torch.utils.bottleneck causes error. How can I avoid this error?

Thanks,

[Arthur151]

Sorry, never use the torch.utils.bottleneck. Don't know why.

[mkhoshle]

@Arthur151 It doesnot seem to be related to torch.utils.bottleneck. When I run (/z/home/mahzad-khosh/env/romp2) mahzad-khosh@prism:/z/home/mahzad-khosh/ROMP/romp$ python -m torch.utils.bottleneck train.py --configs_yml='configs/v1_hrnet_3dpw_ft.yml' I get the following error. It has to do with the way romp is structured.

`bottleneck` is a tool that can be used as an initial step for debugging
bottlenecks in your program.

It summarizes runs of your script with the Python profiler and PyTorch's
autograd profiler. Because your script will be profiled, please ensure that it
exits in a finite amount of time.

For more complicated uses of the profilers, please see
https://docs.python.org/3/library/profile.html and
https://pytorch.org/docs/master/autograd.html#profiler for more information.
Running environment analysis...
Running your script with cProfile
Traceback (most recent call last):
  File "/z/home/mahzad-khosh/env/romp2/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/z/home/mahzad-khosh/env/romp2/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/z/home/mahzad-khosh/env/romp2/lib/python3.8/site-packages/torch/utils/bottleneck/__main__.py", line 229, in <module>
    main()
  File "/z/home/mahzad-khosh/env/romp2/lib/python3.8/site-packages/torch/utils/bottleneck/__main__.py", line 207, in main
    cprofile_prof = run_cprofile(code, globs)
  File "/z/home/mahzad-khosh/env/romp2/lib/python3.8/site-packages/torch/utils/bottleneck/__main__.py", line 75, in run_cprofile
    exec(code, globs, None)
  File "train.py", line 1, in <module>
    from .base import *
ImportError: attempted relative import with no known parent package

[vivi90]

@mkhoshle Please provide more info about your setup.

At least:

• Whick OS?
• Which python version? (python -V)
• Which packages? (pip freeze)

[mkhoshle]

@vivi90 Python version is 3.8.13, Operating system Ubuntu 20.04.3 LTS, Here is the output of the pip freeze:

absl-py==1.0.0
addict==2.4.0
aiohttp==3.8.1
aiosignal==1.2.0
anyio==3.5.0
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.0.5
async-timeout==4.0.2
attrs==21.4.0
autobahn==22.4.2
Automat==20.2.0
Babel==2.10.1
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.0
bvhtoolbox==0.1.3
cachetools==5.0.0
certifi==2020.6.20
cffi==1.15.0
charset-normalizer==2.0.12
chumpy==0.70
colorama==0.4.4
commonmark==0.9.1
constantly==15.1.0
cryptography==37.0.2
cycler==0.11.0
debugpy==1.6.0
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.13
deprecation==2.1.0
distlib==0.3.4
entrypoints==0.4
executing==0.8.3
fastjsonschema==2.15.3
filelock==3.6.0
filterpy==1.4.5
fonttools==4.33.3
freetype-py==2.3.0
frozenlist==1.3.0
future==0.18.2
fvcore==0.1.5.post20220506
google-auth==2.6.6
google-auth-oauthlib==0.4.6
grpcio==1.46.0
h5py==3.6.0
hyperlink==21.0.0
idna==3.3
imageio==2.19.1
imageio-ffmpeg==0.4.7
imgaug==0.4.0
importlib-metadata==4.11.3
importlib-resources==5.7.1
incremental==21.3.0
iopath==0.1.9
ipdb==0.13.9
ipykernel==6.13.0
ipython==8.3.0
ipython-genutils==0.2.0
ipywidgets==7.7.0
jedi==0.18.1
Jinja2==3.1.2
joblib==1.1.0
json5==0.9.8
jsonschema==4.5.1
jupyter-client==7.3.1
jupyter-core==4.10.0
jupyter-packaging==0.11.1
jupyter-server==1.17.0
jupyterlab==3.4.0
jupyterlab-pygments==0.2.2
jupyterlab-server==2.13.0
jupyterlab-widgets==1.1.0
keyboard==0.13.5
kiwisolver==1.4.2
lap==0.4.0
loguru==0.6.0
Markdown==3.3.7
MarkupSafe==2.1.1
matplotlib==3.5.2
matplotlib-inline==0.1.3
mistune==0.8.4
mkl-fft==1.3.1
mkl-random @ file:///tmp/build/80754af9/mkl_random_1626186064646/work
mkl-service==2.4.0
multidict==6.0.2
munkres==1.1.4
nbclassic==0.3.7
nbclient==0.6.3
nbconvert==6.5.0
nbformat==5.4.0
nest-asyncio==1.5.5
networkx==2.8
norfair==0.4.3
notebook==6.4.11
notebook-shim==0.1.0
numpy @ file:///opt/conda/conda-bld/numpy_and_numpy_base_1651563629415/work
numpy-quaternion==2022.4.2
oauthlib==3.2.0
open3d==0.15.2
opencv-contrib-python==4.5.5.64
opencv-python==4.5.5.64
packaging==21.3
pandas==1.4.2
pandocfilters==1.5.0
parso==0.8.3
pbr==5.9.0
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.0.1
platformdirs==2.5.2
plotly==5.7.0
plyfile==0.7.4
portalocker==2.4.0
prettytable==3.3.0
prometheus-client==0.14.1
prompt-toolkit==3.0.29
protobuf==3.20.1
psutil==5.9.0
ptyprocess==0.7.0
pure-eval==0.2.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycocotools==2.0.4
pycparser==2.21
pygame==2.1.2
pyglet==1.5.23
Pygments==2.12.0
PyOpenGL==3.1.0
pyparsing==3.0.8
pyquaternion==0.9.9
pyrender==0.1.45
pyrsistent==0.18.1
python-dateutil==2.8.2
pytorch3d @ https://github.com/Arthur151/ROMP/releases/download/v1.1/pytorch3d-0.6.1-cp38-cp38-linux_x86_64.whl
pytz==2022.1
PyWavelets==1.3.0
PyYAML==5.4.1
pyzmq==22.3.0
requests==2.27.1
requests-oauthlib==1.3.1
rich==9.13.0
rsa==4.8
scikit-image==0.19.2
scikit-learn==1.0.2
scipy==1.8.0
Send2Trash==1.8.0
Shapely==1.8.2
shyaml==0.6.2
six @ file:///tmp/build/80754af9/six_1644875935023/work
smplx==0.1.28
sniffio==1.2.0
soupsieve==2.3.2.post1
stack-data==0.2.0
stevedore==3.5.0
tabulate==0.8.9
tenacity==8.0.1
tensorboard==2.9.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
termcolor==1.1.0
terminado==0.13.3
threadpoolctl==3.1.0
tifffile==2022.5.4
tinycss2==1.1.1
toml==0.10.2
tomlkit==0.10.2
torch==1.10.0+cu102
torchfile==0.1.0
torchvision==0.11.1+cu102
tornado==6.1
tqdm==4.64.0
traitlets==5.1.1
transforms3d==0.3.1
trimesh==3.12.0
Twisted==22.4.0
txaio==22.2.1
typing-extensions==3.10.0.2
urllib3==1.26.9
vedo==2022.2.3
virtualenv==20.14.1
virtualenv-clone==0.5.7
virtualenvwrapper==4.8.4
vtk==9.0.3
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.3.2
Werkzeug==2.1.2
widgetsnbextension==3.6.0
wrapt==1.14.1
wslink==1.6.4
yacs==0.1.8
yarl==1.7.2
zipp==3.8.0
zope.interface==5.4.0

[vivi90]

@mkhoshle

I do python -m torch.utils.bottleneck romp.train --configs_yml='configs/v1_hrnet_3dpw_ft.yml'

The issue is, that according to the documentation the module torch.utils.bottleneck expects an python script file. So using commands doesn't work.

When I run (/z/home/mahzad-khosh/env/romp2) mahzad-khosh@prism:/z/home/mahzad-khosh/ROMP/romp$ python -m torch.utils.bottleneck train.py --configs_yml='configs/v1_hrnet_3dpw_ft.yml' I get the following error. [..] ImportError: attempted relative import with no known parent package

Also calling submodules from inside an package does not necessarily work. That has not '[..] to do with the way romp is structured [..]'. In fact it's bad style to do that. 🙂

So one possible solution for you might be using this wrapper file i have written for you. wrap.py:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# SPDX-License-Identifier: AGPL-3.0-or-later

"""Wrapps call."""

from sys import argv
from subprocess import run  # nosec B404


run(  # nosec B602
    " ".join(argv[1:]),
    shell=True, check=True
)

You are using it by calling:

python -m torch.utils.bottleneck wrap.py -- <command>

So in your case:

python -m torch.utils.bottleneck wrap.py -- romp.train --configs_yml='configs/v1_hrnet_3dpw_ft.yml'

Please close this issue, if it works 🙂