Genesis [FEATURE] Add support for Viewer and headless rendering on windows

This PR initially fixes issue #12. Further tests should be investigated.

Dec 19 '24 17:12 DearVa

I got a numba typeerror. find the output below.

OpenGL.lazywrapper.lazy( 'glGetUniformLocation' )
[Genesis] [18:46:38] [INFO] ╭─────────────────────────────────────────────────────────────────────────────────────╮
[Genesis] [18:46:38] [INFO] │┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉ Genesis ┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉│
[Genesis] [18:46:38] [INFO] ╰─────────────────────────────────────────────────────────────────────────────────────╯
[Genesis] [18:46:38] [INFO] Running on [NVIDIA GeForce GTX 1650 Ti] with backend gs.cuda. Device memory: 4.00 GB.
[Genesis] [18:46:39] [DEBUG] [Taichi] version 1.7.2, llvm 15.0.1, commit 0131dce9, win, python 3.10.4
[Genesis] [18:46:39] [DEBUG] [Taichi] Starting on arch=cuda
[Genesis] [18:46:39] [INFO] 🚀 Genesis initialized. 🔖 version: 0.2.0, 🌱 seed: None, 📏 precision: '32', 🐛 debug: False, 🎨 theme: 'dark'.
[Genesis] [18:46:40] [INFO] Scene <753239f> created.
[Genesis] [18:46:40] [INFO] Adding <gs.PBD3DEntity>. idx: 0, uid: <9c1086f>, morph: <gs.morphs.Mesh(file='C:\Python310\lib\site-packages\genesis\assets\meshes/dragon/dragon.obj')>, material: <gs.PBD.Elastic>.
[Genesis] [18:46:40] [DEBUG] Remeshed file (`.rm`) found in cache.
[Genesis] [18:46:41] [INFO] Building scene <753239f>...
[Genesis] [18:46:43] [INFO] Compiling simulation kernels...
[Genesis] [18:46:50] [INFO] Building visualizer...
Traceback (most recent call last):
  File "E:\GenesisTest\gen.py", line 65, in <module>
    main()
  File "E:\GenesisTest\gen.py", line 56, in main
    scene.build()
  File "C:\Python310\lib\site-packages\genesis\utils\misc.py", line 38, in wrapper
    return method(self, *args, **kwargs)
  File "C:\Python310\lib\site-packages\genesis\engine\scene.py", line 556, in build
    self._visualizer.build()
  File "C:\Python310\lib\site-packages\genesis\vis\visualizer.py", line 101, in build
    self._context.build(self._scene)
  File "C:\Python310\lib\site-packages\genesis\vis\rasterizer_context.py", line 70, in build
    self.jit = JITRenderer(self._scene, [], [])
  File "C:\Python310\lib\site-packages\genesis\ext\pyrender\jit_render.py", line 201, in __init__
    self.gen_func_ptr()
  File "C:\Python310\lib\site-packages\genesis\ext\pyrender\jit_render.py", line 384, in gen_func_ptr
    def forward_pass(
  File "C:\Python310\lib\site-packages\numba\core\decorators.py", line 232, in wrapper
    disp.compile(sig)
  File "C:\Python310\lib\site-packages\numba\core\dispatcher.py", line 905, in compile
    cres = self._compiler.compile(args, return_type)
  File "C:\Python310\lib\site-packages\numba\core\dispatcher.py", line 84, in compile
    raise retval
  File "C:\Python310\lib\site-packages\numba\core\dispatcher.py", line 94, in _compile_cached
    retval = self._compile_core(args, return_type)
  File "C:\Python310\lib\site-packages\numba\core\dispatcher.py", line 107, in _compile_core
    cres = compiler.compile_extra(self.targetdescr.typing_context,
  File "C:\Python310\lib\site-packages\numba\core\compiler.py", line 744, in compile_extra
    return pipeline.compile_extra(func)
  File "C:\Python310\lib\site-packages\numba\core\compiler.py", line 438, in compile_extra
    return self._compile_bytecode()
  File "C:\Python310\lib\site-packages\numba\core\compiler.py", line 506, in _compile_bytecode
    return self._compile_core()
  File "C:\Python310\lib\site-packages\numba\core\compiler.py", line 485, in _compile_core
    raise e
  File "C:\Python310\lib\site-packages\numba\core\compiler.py", line 472, in _compile_core
    pm.run(self.state)
  File "C:\Python310\lib\site-packages\numba\core\compiler_machinery.py", line 368, in run
    raise patched_exception
  File "C:\Python310\lib\site-packages\numba\core\compiler_machinery.py", line 356, in run
    self._runPass(idx, pass_inst, state)
  File "C:\Python310\lib\site-packages\numba\core\compiler_lock.py", line 35, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "C:\Python310\lib\site-packages\numba\core\compiler_machinery.py", line 311, in _runPass
    mutated |= check(pss.run_pass, internal_state)
  File "C:\Python310\lib\site-packages\numba\core\compiler_machinery.py", line 273, in check
    mangled = func(compiler_state)
  File "C:\Python310\lib\site-packages\numba\core\typed_passes.py", line 112, in run_pass
    typemap, return_type, calltypes, errs = type_inference_stage(
  File "C:\Python310\lib\site-packages\numba\core\typed_passes.py", line 93, in type_inference_stage
    errs = infer.propagate(raise_errors=raise_errors)
  File "C:\Python310\lib\site-packages\numba\core\typeinfer.py", line 1091, in propagate
    raise errors[0]
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Unknown attribute 'glUseProgram' of type GLFunc

File "C:\Python310\lib\site-packages\genesis\ext\pyrender\jit_render.py", line 420:
        def forward_pass(
            <source elided>
                if pid != last_pid:
                    gl.glUseProgram(pid)
                    ^

During: typing of get attribute at C:\Python310\lib\site-packages\genesis\ext\pyrender\jit_render.py (420)

File "C:\Python310\lib\site-packages\genesis\ext\pyrender\jit_render.py", line 420:
        def forward_pass(
            <source elided>
                if pid != last_pid:
                    gl.glUseProgram(pid)
                    ^

[Genesis] [18:47:04] [INFO] 💤 Exiting Genesis and caching compiled kernels...

Dec 19 '24 17:12 agenel

This PR initially fixes issue #12. Further tests should be investigated.

Thank you for your reply,but when i modify the code,the other question come. the code below call cam.render(), for i in range(110): scene.step() cam.set_pose( pos = (3.0 * np.sin(i / 60), 3.0 * np.cos(i / 60), 2.5), lookat = (0, 0, 0.5), ) cam.render() cam.stop_recording(save_to_filename='video.mp4', fps=60)

#cam.render() will call render_offscreen function in viewer.py

#wait_for_offscreen self._offscreen_result_semaphore.acquire()

this part will block the thread,how can i fix it

Dec 20 '24 02:12 lixiaoming0017

(som) PS E:\code\Genesis_windows> python .\run.py
[Genesis] [14:34:16] [INFO] ╭─────────────────────────────────────────────────────────────────────────────────────╮
[Genesis] [14:34:16] [INFO] │┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉ Genesis ┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉│
[Genesis] [14:34:16] [INFO] ╰─────────────────────────────────────────────────────────────────────────────────────╯
[Genesis] [14:34:16] [INFO] Running on [NVIDIA GeForce RTX 3080 Laptop GPU] with backend gs.cuda. Device memory: 16.00 GB.
[Genesis] [14:34:16] [INFO] 🚀 Genesis initialized. 🔖 version: 0.2.0, 🌱 seed: None, 📏 precision: '32', 🐛 debug: False, 🎨 theme: 'dark'.
[Genesis] [14:34:17] [INFO] Scene <b6b2153> created.
[Genesis] [14:34:17] [INFO] Adding <gs.RigidEntity>. idx: 0, uid: <4f28967>, morph: <gs.morphs.Plane>, material: <gs.materials.Rigid>.
[Genesis] [14:34:17] [INFO] Adding <gs.RigidEntity>. idx: 1, uid: <75175ca>, morph: <gs.morphs.MJCF(file='E:\code\Genesis_windows\genesis\assets\xml/franka_emika_panda/panda.xml')>, material: <gs.materials.Rigid>.
[Genesis] [14:34:17] [INFO] Building scene <b6b2153>...
[Genesis] [14:34:41] [INFO] Compiling simulation kernels...
[Genesis] [14:35:07] [INFO] Building visualizer...
Traceback (most recent call last):
  File "E:\code\Genesis_windows\genesis\ext\pyrender\numba_gl_wrapper.py", line 71, in load_func
    func_ptr = GL.platform.ctypesloader.buildFunction(
  File "E:\Anaconda\envs\som\lib\site-packages\OpenGL\platform\ctypesloader.py", line 96, in buildFunction
    return functionType( (name, dll), )
AttributeError: function 'glGetUniformLocation' not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\code\Genesis_windows\run.py", line 40, in <module>
    scene.build()
  File "E:\code\Genesis_windows\genesis\utils\misc.py", line 38, in wrapper
    return method(self, *args, **kwargs)
  File "E:\code\Genesis_windows\genesis\engine\scene.py", line 565, in build
    self._visualizer.build()
  File "E:\code\Genesis_windows\genesis\vis\visualizer.py", line 101, in build
    self._context.build(self._scene)
  File "E:\code\Genesis_windows\genesis\vis\rasterizer_context.py", line 70, in build
    self.jit = JITRenderer(self._scene, [], [])
  File "E:\code\Genesis_windows\genesis\ext\pyrender\jit_render.py", line 201, in __init__
    self.gen_func_ptr()
  File "E:\code\Genesis_windows\genesis\ext\pyrender\jit_render.py", line 354, in gen_func_ptr
    self.gl = GLWrapper()
  File "E:\code\Genesis_windows\genesis\ext\pyrender\numba_gl_wrapper.py", line 41, in __init__
    load_func("glGetUniformLocation", GLint, GLuint, GLvoidp)
  File "E:\code\Genesis_windows\genesis\ext\pyrender\numba_gl_wrapper.py", line 78, in load_func
    func_ptr = glplatform.createExtensionFunction(
NameError: name 'glplatform' is not defined. Did you mean: 'GLplatform'?
[Genesis] [14:35:09] [INFO] 💤 Exiting Genesis and caching compiled kernels...

Thanks authors for the quick correction! the typo is solved.

Dec 20 '24 06:12 guanjunwu

(som) PS E:\code\Genesis_windows> python .\run.py
[Genesis] [16:29:11] [INFO] ╭─────────────────────────────────────────────────────────────────────────────────────╮
[Genesis] [16:29:11] [INFO] │┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉ Genesis ┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉│
[Genesis] [16:29:11] [INFO] ╰─────────────────────────────────────────────────────────────────────────────────────╯
[Genesis] [16:29:11] [INFO] Running on [NVIDIA GeForce RTX 3080 Laptop GPU] with backend gs.cuda. Device memory: 16.00 GB.
[Genesis] [16:29:11] [INFO] 🚀 Genesis initialized. 🔖 version: 0.2.0, 🌱 seed: None, 📏 precision: '32', 🐛 debug: False, 🎨 theme: 'dark'.
[Genesis] [16:29:12] [INFO] Scene <480ae95> created.
[Genesis] [16:29:12] [INFO] Adding <gs.RigidEntity>. idx: 0, uid: <1d12ad9>, morph: <gs.morphs.Plane>, material: <gs.materials.Rigid>.
[Genesis] [16:29:12] [INFO] Adding <gs.RigidEntity>. idx: 1, uid: <0328e4b>, morph: <gs.morphs.MJCF(file='E:\code\Genesis_windows\genesis\assets\xml/franka_emika_panda/panda.xml')>, material: <gs.materials.Rigid>.
[Genesis] [16:29:12] [INFO] Building scene <480ae95>...
[Genesis] [16:29:31] [INFO] Compiling simulation kernels...
[Genesis] [16:29:55] [INFO] Building visualizer...
[Genesis] [16:29:58] [INFO] Viewer created. Resolution: 1280×960, max_FPS: 60.
0
[Genesis] [16:29:58] [INFO] Running at 120.79 FPS.

Hi, I found that my program will stuck at here, and no viewer displayed. I cannot even kill the script. Why?

Dec 20 '24 08:12 guanjunwu

Thanks, My program is running normally, but no window pops up.

Dec 20 '24 08:12 CaesarAaron

Windows 11 Pro for Workstation Python 3.12.7

I do:

python -m venv venv
venv\Scripts\activate.bat
pip install genesis-world
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
pip instal ffmpeg

(and edited the files to implement this PR)

Then I run a basic script: https://genesis-world.readthedocs.io/en/latest/user_guide/getting_started/hello_genesis.html

venv\Scripts\activate.bat
python test.py

The simulation works, but does not show the viewer window.

What am I doing wrong?

Dec 20 '24 09:12 Sipovec

is no viewer rendering support added to windows

Dec 30 '24 10:12 sridhar-mani

They have plan to add viewer rendering on Windows ?

Jan 06 '25 17:01 RobertPastiss

i dont think so. because i can see they are making improvements to the engine but not the viewing. Guessing this might be abandoned after a while. Just a test release to see the market.

Jan 07 '25 06:01 sridhar-mani

i dont think so. because i can see they are making improvements to the engine but not the viewing. Guessing this might be abandoned after a while. Just a test release to see the market.

We definitely have a plan! Currently a bit short in manpower but we will come back with full thrust in early Feb to start adding this We are also rebuilding the whole rendering pipeline

Jan 07 '25 07:01 zhouxian

Good to know!!! The render issue has been solved in this pull request at least the running part. I have run and tested it. Would be good if you could tell if there is a way to put a windows renderer or send the data through api and render in browser.

Jan 07 '25 10:01 sridhar-mani

你好，我用了您的分支代码，在win11上确实不会报错了，但是也没有渲染窗口弹出，该如何解决呢？

Jan 10 '25 02:01 shaoyidi

yeah thats what i mentioned it is solving the error of running in windows but giving a render window is still now resolved

Jan 10 '25 06:01 sridhar-mani

i dont think so. because i can see they are making improvements to the engine but not the viewing. Guessing this might be abandoned after a while. Just a test release to see the market.

We definitely have a plan! Currently a bit short in manpower but we will come back with full thrust in early Feb to start adding this We are also rebuilding the whole rendering pipeline

So, the viewer for Windows is on the pipe ?

Jan 10 '25 12:01 RobertPastiss

we could also try it if you guys could help a bit actually. @Genesis-Embodied-AI

Jan 11 '25 16:01 sridhar-mani

we could also try it if you guys could help a bit actually. @Genesis-Embodied-AI

Yes it would be extremely helpful if you guys can help make it work!

Jan 11 '25 16:01 zhouxian

yeah but if you could point us in the direction with a bit comments here and there regarding the bug fixes we can make it work.

Jan 11 '25 17:01 sridhar-mani

@DearVa Seems this PR may have been superseded by https://github.com/Genesis-Embodied-AI/Genesis/pull/610 ?

Jan 22 '25 12:01 lgleim

This PR has been superseded by https://github.com/Genesis-Embodied-AI/Genesis/pull/782. Additional camera support is still partially broken though (python crashes). A fix is on the way. Closing this PR.

Feb 26 '25 18:02 duburcqa

Genesis Genesis copied to clipboard

[FEATURE] Add support for Viewer and headless rendering on windows

Genesis
Genesis copied to clipboard