Implicit3DUnderstanding
Implicit3DUnderstanding copied to clipboard
chengzhag
Error in building the project
Hi @chengzhag ,
I got the following error while building the project. Seems like there are some missing files.
Building pyrender...
Run: cd external/mesh_fusion/librender && /home/user/anaconda3/envs/pytorch11_py3_Im3D/bin/python setup.py build_ext --inplace
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /home/user/anaconda3/envs/pytorch11_py3_Im3D/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1832:0,
from /home/user/anaconda3/envs/pytorch11_py3_Im3D/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /home/user/anaconda3/envs/pytorch11_py3_Im3D/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from pyrender.cpp:531:
/home/user/anaconda3/envs/pytorch11_py3_Im3D/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it with " \
^
In file included from pyrender.cpp:534:0:
offscreen.h:4:21: fatal error: GL/glew.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
Traceback (most recent call last):
File "project.py", line 158, in <module>
globals()[args.work](args)
File "project.py", line 120, in build
subprocess.check_output(cmd, shell=True)
File "/home/user/anaconda3/envs/pytorch11_py3_Im3D/lib/python3.6/subprocess.py", line 356, in check_output
**kwargs).stdout
File "/home/user/anaconda3/envs/pytorch11_py3_Im3D/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'cd external/mesh_fusion/librender && /home/user/anaconda3/envs/pytorch11_py3_Im3D/bin/python setup.py build_ext --inplace' returned non-zero exit status 1.
It seems there is no directory called GL. Hence, any reference to headers included in it would probably throw an error.
Would appreciate your help with this.
~Cheers!
Hi @metro-smiles,
It seems like that your OpenGL is not installed properly.
The installation of the libraries required should be done in https://github.com/chengzhag/Implicit3DUnderstanding/blob/af2964f074d941cd990cff81a9b5f75489586ed2/external/build_gaps.sh#L37 when building the project, which should ask for user password and requires higher privilege.
Maybe check if you are running with a user account with the privilege to run apt with sudo?
Hi @chengzhag,
Really appreciate your response. So it did install the graphics libraries after having asked for sudo privileges. So not sure what went wrong. I did not get any error message while the apt-get packages were installing.
Hi @metro-smiles,
Sorry for the inconvenience. My system is Ubuntu 18. I noticed that the error might be caused by the missing library freeglut3-dev and libglew-dev. Could you please try apt install freeglut3-dev libglew-dev as in https://github.com/chengzhag/Implicit3DUnderstanding/tree/af2964f074d941cd990cff81a9b5f75489586ed2/external/mesh_fusion ?
If the problem is solved, I will try to add this line to the project building.
Really appreciate your feedback!
Hi @chengzhag,
Really appreciate your response.
So I have been trying to run on Ubuntu 16, so wonder if that's causing some incompatibility. But anyways, after installing your suggested packages now I'm getting a different error (from CUDA):
Building pyfusion...
Run: cd external/mesh_fusion/libfusiongpu && mkdir -p build && cd build && cmake .. && make && cd .. && /home/user/anaconda3/envs/pytorch11_py3_Im3D/bin/python setup.py build_ext --inplace
CMake Warning (dev) in CMakeLists.txt:
No project() command is present. The top-level CMakeLists.txt file must
contain a literal, direct call to the project() command. Add a line of
code such as
project(ProjectName)
near the top of the file, but after cmake_minimum_required().
CMake is pretending there is a "project(Project)" command on the first
line.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Error at /home/user/anaconda3/envs/pytorch11_py3_Im3D/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
Could NOT find CUDA (missing: CUDA_INCLUDE_DIRS) (found suitable version
"9.0", minimum required is "6.5")
Call Stack (most recent call first):
/home/user/anaconda3/envs/pytorch11_py3_Im3D/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
/home/user/anaconda3/envs/pytorch11_py3_Im3D/share/cmake-3.16/Modules/FindCUDA.cmake:1104 (find_package_handle_standard_args)
CMakeLists.txt:34 (find_package)
Traceback (most recent call last):
File "project.py", line 158, in <module>
globals()[args.work](args)
File "project.py", line 112, in build
subprocess.check_output(cmd, shell=True)
File "/home/user/anaconda3/envs/pytorch11_py3_Im3D/lib/python3.6/subprocess.py", line 356, in check_output
**kwargs).stdout
File "/home/user/anaconda3/envs/pytorch11_py3_Im3D/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'cd external/mesh_fusion/libfusiongpu && mkdir -p build && cd build && cmake .. && make && cd .. && /home/user/anaconda3/envs/pytorch11_py3_Im3D/bin/python setup.py build_ext --inplace' returned non-zero exit status 1.
Hi @metro-smiles,
It looks like that libfusiongpu of mesh_fusion didn't find on your system. If data processing speed is not a problem, you can try to change 'libfusiongpu' in this line https://github.com/chengzhag/Implicit3DUnderstanding/blob/af2964f074d941cd990cff81a9b5f75489586ed2/project.py#L103 into 'libfusioncpu'. Or try to install CUDA NVCC.
I'll add the requirement to README if it is the case.
Hi @chengzhag ,
Really appreciate all your help/inputs! I was able to build and run the demo.
To summarize, following were the changes I ended having to make:
- Install these libraries, separately:
$ apt install freeglut3-dev libglew-dev
- Ensure nvcc is callable from command-line
- If nvcc is not available, lifusioncpu can instead be used for build_gaps but nvcc will anyways be needed for ldif2mesh
- Incase, gcc compiler is not set to C++ 2011 standard, set --std c++11 in the nvcc command in ldif2mesh/build.sh
Hi @chengzhag ,
Apologies for re-opening the issue.
So while pre-processing the Pix3D dataset, it seems that the mesh_fusion/fusion.py code is not really using the GPU, eventhough I compiled the project with 'libfusiongpu' and set the use_gpu flag to True. Can you please suggest some possible explanation for this?
Btw, the demo component seems to work fine.
Appreciate your inputs.
Hi @metro-smiles,
From what I understand, the mesh_fusion/fusion.py does import libfusiongpu when use_gpu set to True. But further experiments indicate that GPU is truely not used (with the help of nvidia-smi) and there is no speed difference between libfusiongpu and libfusioncpu. It seems like that there is a problem with mesh_fusion lib, which I did not notice and cannot provide further help.
However, the preprocessing of Pix3D is accelerated with multi-thread, which can be done in less than hour. Thus the problem seems not very significant.
Thanks for your summary for the compiling problem. We updated two recent commits to solving it.
Hi @chengzhag ,
Appreciate your inputs.
But it seems that the Pix3D pre-processing step is not as fast, at my end. I've tried it independently on two different systems but on both it's been running for over a day without making any progress. (It seems that call to fusion.py with mode=render is the most expensive and like I was saying before it's not using the GPU.)
Output of htop:
Is there any chance you pre-processed it differently / did some other intermediate step which is not mentioned in the README?
~Cheers!
Hi @metro-smiles,
I just recall that Pix3D pre-processing consists of two steps (i.e. image processing and mesh processing). Sorry that I wrongly think of the time consumption of image processing as the total pre-processing time.
Also my bad that I missed meshlab installation in the README, which you probably have found out or pre-installed.
As for the mesh processing step, I tried to run it with libfusiongpu, finding out that it actually has no problem using GPU:
I think It might be the case that the program opened too many processes for the machine to handle. Could you please try to run preprocess_pix3d4ldif with processes flag set to 0?
(Hint: You can comment out the lines between print('Processing imgs...') and print('Processing meshs...') to skip image processing.)
Here is an example when I set processes to 0:
If it is the case, I will disable multi-thread for mesh processing.
Hi @chengzhag ,
So I don't know why but the check with the main in fusion.py/scale.py/etc. was returning false. Hence, I had to turn of the subprocess call and instead had to instantiate an object of those classes and then call the run method from the object instance. Like the lines you've commented out just prior to the subprocess call.
Hi @metro-smiles,
That's weird. The reason that subprocess.check_output() is used to call fusion.py/scale.py/etc. is to run them off-screen with xvfb-run. I think these two kinds of calling methods should be identical if you are running on a screen.
I presume that you fixed the problem of not running on GPU and the processing speed with such changes? May I ask where did you get False returns from specifically? Maybe somewhere near below? https://github.com/chengzhag/Implicit3DUnderstanding/blob/262da05928fedeb5ba95983c3dea121ca73e712b/external/mesh_fusion/fusion.py#L14
Glad to know that your problems have been solved and thanks for your feedback.
For the later comers if you have encountered the same problem: I'll look into the problem but by far the best way of solving the problem might be following @metro-smiles to directly import fusion.py/scale.py/etc. and use their class.
Hi @chengzhag, sorry for reopening the issue again but I got problem when building project, as well.
I tried the solutions here and It seems a different error.
whether the 37th line in build_gap.sh is commented out or not, when I run the python project.py build, I got this:
# Ptsview and qview aren't built by default, adjust the makefile to build them.
# sed commands are for script idempotency
sed -i.bak '/ptsview/d' ./apps/Makefile
sed -i.bak '/qview/d' ./apps/Makefile
echo " cd ptsview; \$(MAKE) \$(TARGET)" >> ./apps/Makefile
echo " cd qview; \$(MAKE) \$(TARGET)" >> ./apps/Makefile
# Make GAPS (assuming 8 threads):
# On MacOS, using OSMesa is more difficult, so we don't
if [[ ! $(uname -s) == Darwin* ]]
then
make mesa -j8
else
make -j8
fi
/usr/bin/ld: cannot find -lGL
collect2: error: ld returned 1 exit status
make[5]: *** [../../bin/x86_64/img2img] Error 1
make[4]: *** [mesa] Error 2
make[3]: *** [target] Error 2
make[2]: *** [mesa] Error 2
make[1]: *** [target] Error 2
make: *** [mesa] Error 2
Traceback (most recent call last):
File "project.py", line 158, in <module>
globals()[args.work](args)
File "project.py", line 97, in build
subprocess.check_output(cmd, shell=True)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/subprocess.py", line 356, in check_output
**kwargs).stdout
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
when I run the sudo python project.py build, I got this:
File "project.py", line 17
runs = api.runs(f'{args.username}/{args.project}')
^
SyntaxError: invalid syntax
I checked out the code there isn't any invalid syntax.
Appreciate your help ^^
Hi @emirhanKural,
When running python project.py build, there is no need to comment out the 37th line in build_gap.sh. It will ask for sudo privilege and install essential packages with apt-get.
The first error seems like there is a certain missing package, which might be caused by the failure of installation in the previous steps. Please make sure you are running with a user with sudo privilege to run apt. Alternatively, you can ask your administrator for installation of the packages in the 37th line in build_gap.sh and comment out the line.
I have no clue about the second error, but it might be caused by python project.py build running with sudo. Again, there is no need to run with sudo, the program will ask for sudo privilege when running apt-get.
Hi @emirhanKural, thanks for your response.
But I still have the issue. I tried to start an clean installation but /usr/bin/ld: cannot find -lGL shows itself again.
I installed libraries you mentioned above manually, as well.
It's like getting header/library problem.
# Ptsview and qview aren't built by default, adjust the makefile to build them.
# sed commands are for script idempotency
sed -i.bak '/ptsview/d' ./apps/Makefile
sed -i.bak '/qview/d' ./apps/Makefile
echo " cd ptsview; \$(MAKE) \$(TARGET)" >> ./apps/Makefile
echo " cd qview; \$(MAKE) \$(TARGET)" >> ./apps/Makefile
# Make GAPS (assuming 8 threads):
# On MacOS, using OSMesa is more difficult, so we don't
if [[ ! $(uname -s) == Darwin* ]]
then
make mesa -j8
else
make -j8
fi
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libRNBasics.a
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libRNMath.a
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libR2Shapes.a
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libR3Shapes.a
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libR3Graphics.a
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libR3Surfels.a
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libRGBD.a
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libfglut.a
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libjpeg.a
ar: `u' modifier ignored since `D' is the default (see `U')
ar: creating ../../lib/x86_64/libpng.a
/usr/bin/ld: cannot find -lGL
collect2: error: ld returned 1 exit status
make[5]: *** [../../bin/x86_64/img2img] Error 1
make[4]: *** [mesa] Error 2
make[3]: *** [target] Error 2
make[2]: *** [mesa] Error 2
make[1]: *** [target] Error 2
make: *** [mesa] Error 2
Traceback (most recent call last):
File "project.py", line 158, in <module>
globals()[args.work](args)
File "project.py", line 97, in build
subprocess.check_output(cmd, shell=True)
File "/home/ubuntu/anaconda3/envs/Im3D/lib/python3.6/subprocess.py", line 356, in check_output
**kwargs).stdout
File "/home/ubuntu/anaconda3/envs/Im3D/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'cd external && bash build_gaps.sh' returned non-zero exit status 2.
Hi @emirhanKural,
It seems that there are some comments in the .sh file provided by gaps about opengl .so file. Maybe check if your condition matches the description of the comments?
Also, did you follow README to install others packeges (by sudo apt install xvfb ninja-build freeglut3-dev libglew-dev meshlab) which includes one associated to opengl?
Hello @chengzhag, thank you for response ^^
The problem was solved by this comment. Changing /usr/lib/ symbolic link to newest nvidia libGL link worked.
Firstly, by locate libGL.so see links.
Then, pick the newest cuda/nvidia libGL link and add it in /usr/lib/
sudo ln -sf /usr/local/cuda-11.1/nsight-systems-2020.3.4/host-linux-x64/Mesa/libGL.so.1.5.0 /usr/lib/libGL.so
I could build the project. I think we can close this issue.
However I want to ask something about demo running. But when I tried to run demo.py, I got Failed to extract mesh with error ValueError. I got outputs like recon and d3dbbox images and objes but not sure it's worked well. Any idea ?
Begin to finetune from the existing weight.
Loading checkpoint from out/total3d/21070510413690/model_best.pth.
set() subnet missed.
Weights for finetuning loaded.
----------------------------------------------------------------------------------------------------
Loading data.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Failed to extract mesh with error ValueError('Surface level must be within volume data range.',). Setting to unit sphere.
Time elapsed: 10.886757135391235.
Hi @emirhanKural,
Glad that you solved the problem!
About the second problem, could you please check issue #1 to see if the solution applies to you?
Hope this can help.
Hi @chengzhag,
The solution of issue #1 worked by cuda=False.
But I think it run very slowly, time elapsed is almost 40-70 sec. Is there any way to use cuda=True or speed up the prediction.
Really thank you so much for your responses ^^
Hi @emirhanKural,
The reason for this problem might be related to the incompatibility between the Cuda version setting used when compiling and the actual Cuda version used when running. You may need to check if there is any difference between the version used when compiling and running.
We've tried our best to provide support for compiling for Cuda 8, 9, 10 in this commit. Cuda 11 is also later supported.
If the problem persists, sorry that I can not provide any further support since the Cuda kernel is provided by LDIF.
Hope the information can help.
Hello @chengzhag,
I try to go on with cuda=False for now. I again really appreciate for your help.
I don't want to bother you any more, but I want to ask one more thing ^^
Where detections.json is come from? Do you use a model, which is not included in this project ?
Cuz I cannot give only one image and not get outputs directly :)
If you use a model or a network, can you share with me or similar something ?
Thank you for your responses!
Hi @emirhanKural,
We use the detection results provided by Total3D and Coop. However, the 2D detector is not included in their code.
The detections.json of the demo is converted from the provided detections. If you want to test on your own data, you have to either fine-tune a 2D detector on SUN-RGBD or fine-tune it on your own dataset and map the output categories.
`{'bbox_half': 0.7, 'spacing': 0.04375} Processing imgs... 55%|████████████████████████████████████████████████████████████████████████████████▋ | 5565/10069 [06:22<03:17, 22.81it/s]/home/caoxingwen/anaconda3/envs/Im3Done/lib/python3.9/site-packages/PIL/Image.py:960: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images warnings.warn( 84%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍ | 8446/10069 [14:10<09:15, 2.92it/s]/home/caoxingwen/anaconda3/envs/Im3Done/lib/python3.9/site-packages/PIL/Image.py:960: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images warnings.warn( 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10069/10069 [22:44<00:00, 7.38it/s] Processing meshs... 0%| | 0/735 [01:16<?, ?it/s] multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/home/caoxingwen/anaconda3/envs/Im3Done/lib/python3.9/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/mnt/d/OneDrive_caoxingwen/3d-layout-code/Implicit3DUnderstanding-main2/Implicit3DUnderstanding-main/utils/preprocess_pix3d4ldif.py", line 143, in process_mesh watertight_obj = make_watertight(normalized_obj, output_folder) File "/mnt/d/OneDrive_caoxingwen/3d-layout-code/Implicit3DUnderstanding-main2/Implicit3DUnderstanding-main/utils/preprocess_pix3d4ldif.py", line 79, in make_watertight subprocess.check_output(f'xvfb-run -a -s "-screen 0 800x600x24" {python_bin} {mesh_fusion}/fusion.py' File "/home/caoxingwen/anaconda3/envs/Im3Done/lib/python3.9/subprocess.py", line 424, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/caoxingwen/anaconda3/envs/Im3Done/lib/python3.9/subprocess.py", line 528, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'xvfb-run -a -s "-screen 0 800x600x24" /home/caoxingwen/anaconda3/envs/Im3Done/bin/python external/mesh_fusion/fusion.py --mode=render --in_file data/pix3d/ldif/bed/IKEA_BEDDINGE.model/mesh_orig.off --out_dir data/pix3d/ldif/bed/IKEA_BEDDINGE.model --overwrite' returned non-zero exit status 1. """
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/mnt/d/OneDrive_caoxingwen/3d-layout-code/Implicit3DUnderstanding-main2/Implicit3DUnderstanding-main/utils/preprocess_pix3d4ldif.py", line 225, in
The following error occurs when I run the preprocess_pix3d4ldif.py file. Could you please help me to solve it？
Hi @CAOXINGWEN,
Could you try to run this command xvfb-run -a -s "-screen 0 800x600x24" /home/caoxingwen/anaconda3/envs/Im3Done/bin/python external/mesh_fusion/fusion.py --mode=render --in_file data/pix3d/ldif/bed/IKEA_BEDDINGE.model/mesh_orig.off --out_dir data/pix3d/ldif/bed/IKEA_BEDDINGE.model --overwrite from your project folder? This may help you to identify the problem.
Hi @CAOXINGWEN,
Could you try to run this command
xvfb-run -a -s "-screen 0 800x600x24" /home/caoxingwen/anaconda3/envs/Im3Done/bin/python external/mesh_fusion/fusion.py --mode=render --in_file data/pix3d/ldif/bed/IKEA_BEDDINGE.model/mesh_orig.off --out_dir data/pix3d/ldif/bed/IKEA_BEDDINGE.model --overwritefrom your project folder? This may help you to identify the problem.
This error occurred when I entered this command at your suggestion？
Hi @CAOXINGWEN,
Haven’t encountered this problem before. Maybe run without xvfb-run like /home/caoxingwen/anaconda3/envs/Im3Done/bin/python external/mesh_fusion/fusion.py --mode=render --in_file data/pix3d/ldif/bed/IKEA_BEDDINGE.model/mesh_orig.off --out_dir data/pix3d/ldif/bed/IKEA_BEDDINGE.model --overwrite to see if the error has anything to do with it?
Note: You will need to run this with a screen.
If the same error happens, try to run this in debug mode, for example with PyCharm, to see what part of the code caused the Segmentation fault error?
Note: Since the error happens in the third party mesh_fusion tool, sorry that I can not provide detailed support. You may refer to its own README to follow their steps for a manual compilation and demo running to check if the module is working as intended.