Feature request: please add an example working in Google Colab

[vak]

I failed to find the way to run Genesis in Google Colab Notebook.

import genesis as gs
gs.init(backend=gs.cpu)

scene = gs.Scene(show_viewer=True)
plane = scene.add_entity(gs.morphs.Plane())
franka = scene.add_entity(
    gs.morphs.MJCF(file='xml/franka_emika_panda/panda.xml'),
)

scene.build()

for i in range(1000):
    scene.step()

An error I am getting is:

[Genesis] [14:25:35] [INFO] ╭───────────────────────────────────────────────╮
INFO:genesis:~<╭───────────────────────────────────────────────╮>~
[Genesis] [14:25:35] [INFO] │┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈ Genesis ┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈│
INFO:genesis:~<│┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈>~ ~~~~<Genesis>~~~~ ~<┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈┉┈│>~
[Genesis] [14:25:35] [INFO] ╰───────────────────────────────────────────────╯
INFO:genesis:~<╰───────────────────────────────────────────────╯>~
[Genesis] [14:25:35] [INFO] Running on [13th Gen Intel(R) Core(TM) i7-13700T] with backend gs.cpu. Device memory: 62.50 GB.
INFO:genesis:Running on ~~<[13th Gen Intel(R) Core(TM) i7-13700T]>~~ with backend ~~<gs.cpu>~~. Device memory: ~~<62.50>~~ GB.
[Genesis] [14:25:35] [INFO] 🚀 Genesis initialized. 🔖 version: 0.2.0, 🌱 seed: None, 📏 precision: '32', 🐛 debug: False, 🎨 theme: 'dark'.
INFO:genesis:🚀 Genesis initialized. 🔖 version: ~~<0.2.0>~~, 🌱 seed: ~~<None>~~, 📏 precision: '~~<32>~~', 🐛 debug: ~~<False>~~, 🎨 theme: '~~<dark>~~'.
---------------------------------------------------------------------------
GenesisException                          Traceback (most recent call last)
<ipython-input-4-39f14d971592> in <cell line: 4>()
      2 gs.init(backend=gs.cpu)
      3 
----> 4 scene = gs.Scene(show_viewer=True)
      5 plane = scene.add_entity(gs.morphs.Plane())
      6 franka = scene.add_entity(

3 frames
/usr/local/lib/python3.10/dist-packages/genesis/utils/misc.py in raise_exception(msg)
     16 def raise_exception(msg="Something went wrong."):
     17     gs.logger._error_msg = msg
---> 18     raise gs.GenesisException(msg)
     19 
     20 

GenesisException: No display detected. Use `show_viewer=False` for headless mode.

Hopefully it could work in Google Colab though

[mohamedfadlalla]

try this code:

import genesis as gs gs.init(backend=gs.cuda) # Keeping CPU backend as in your original code

scene = gs.Scene( show_viewer=False, # Disable viewer since we're recording # renderer=gs.renderers.Rasterizer( # Configure rasterizer for better performance # max_samples_per_pixel=1 # Reduce samples for faster rendering # ) )

Add entities

plane = scene.add_entity(gs.morphs.Plane()) franka = scene.add_entity( gs.morphs.MJCF(file='xml/franka_emika_panda/panda.xml'), )

Add camera with lower resolution

cam = scene.add_camera( res=(320, 240), # Reduced resolution for better performance pos=(3.5, 0.0, 2.5), lookat=(0, 0, 0.5), fov=30, GUI=False )

scene.build()

Start recording

cam.start_recording()

Reduce number of frames for testing

for i in range(5): # Reduced from 1000 to 100 frames scene.step()

# Render only every 2nd frame to improve performance
if i % 2 == 0:
    cam.render()

Save with lower FPS

cam.stop_recording(save_to_filename='simulation.mp4', fps=30)

[zswang666]

Other than @mohamedfadlalla answer, you can also do the following before running the script,

export DISPLAY=:1 # the number may vary

The number 1 may vary based on your machine. You need to login the machine with the X display and do echo $DISPLAY

[sudhir2016]

Tried the above code in Colab. The building visualizer step seems to go on for ever. How long should it take ?

[vak]

@mohamedfadlalla many thanks, your recipe worked out!

@sudhir2016 I run Colab locally on my RTX 4090 using public Google's Colab docker image. And scene.build() took 3 minutes.

Rendering of 5 frames like this:

in range(5): scene.step() cam.render() # i.e. without skipping the render() invokations

took 4 minutes.

peak VRAM usage ~1.5 GB (of 24 GB available).

I wonder what happens if scene will be somewhat more complex

[brandonscottbarney]

Python 3 Google Compute Engine backend (GPU) Showing resources from 3:37 PM to 4:25 PM System RAM 3.9 / 51.0 GB

GPU RAM 0.2 / 15.0 GB

Disk 34.0 / 235.7 GB

I'm running this code to download the file at the end:

import genesis as gs gs.init(backend=gs.gpu)

scene = gs.Scene( show_viewer = False, viewer_options = gs.options.ViewerOptions( res = (1280, 960), camera_pos = (3.5, 0.0, 2.5), camera_lookat = (0.0, 0.0, 0.5), camera_fov = 40, max_FPS = 60, ), vis_options = gs.options.VisOptions( show_world_frame = True, world_frame_size = 1.0, show_link_frame = False, show_cameras = False, plane_reflection = True, ambient_light = (0.1, 0.1, 0.1), ), renderer=gs.renderers.Rasterizer(), )

plane = scene.add_entity( gs.morphs.Plane(), ) franka = scene.add_entity( gs.morphs.MJCF(file='xml/franka_emika_panda/panda.xml'), )

cam = scene.add_camera( res = (640, 480), pos = (3.5, 0.0, 2.5), lookat = (0, 0, 0.5), fov = 30, GUI = False, )

scene.build()

render rgb, depth, segmentation, and normal

rgb, depth, segmentation, normal = cam.render(rgb=True, depth=True, segmentation=True, normal=True)

cam.start_recording() import numpy as np

for i in range(120): scene.step() cam.set_pose( pos = (3.0 * np.sin(i / 60), 3.0 * np.cos(i / 60), 2.5), lookat = (0, 0, 0.5), ) cam.render() cam.stop_recording(save_to_filename='video.mp4', fps=60) from google.colab import files files.download('video.mp4')

How long should it run?

Here is the cell where it has been running for 42 minutes:

https://colab.research.google.com/drive/14o013BLkO4AMXY_TgiAqSjiYb8nebury#scrollTo=eklMZgLPSPiA

The entire CoLab is open: https://colab.research.google.com/drive/14o013BLkO4AMXY_TgiAqSjiYb8nebury?usp=sharing

I have a Pro Account and wanted to see if I can get the simulator going but the advertised speedup hasn't seemed to materialize for me as the demo seems to be taking a bit longer to generate than I anticipated. Please advise.