The requested array has an inhomogeneous shape after 1 dimensions

[ytx0323]

Hi, I am trying to train a controller using the code

cd controller python train_act.py \ env=rlbench \ env.dataset_root=/tmp/train_data_rnd_bg/ \ work_dir=/tmp/controller \ demos=25 \ env.train_tasks=[take_lid_off_saucepan] \ num_train_epochs=1000 \ action_sequence=20 \ batch_size=8 \ method.lr=1e-5 \ wandb.use=true

but I encounter a error as below File "/usr/app/genima/controller/train_act.py", line 302, in main workspace.train() File "/usr/app/genima/controller/train_act.py", line 268, in train self._load_demos() File "/usr/app/genima/robobase/robobase/workspace.py", line 490, in _load_demos self.env_factory.load_demos_into_replay( File "/usr/app/genima/controller/env/rlbench.py", line 351, in load_demos_into_replay add_demo_to_replay_buffer(demo_env, buffer) File "/usr/app/genima/controller/env/rlbench_utils.py", line 269, in add_demo_to_replay_buffer replay_buffer.add_final(final_obs) File "/usr/app/genima/robobase/robobase/replay_buffer/uniform_replay_buffer.py", line 435, in add_final episode[k] = np.array(v, self._storage_signature[k].type) ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (5,) + inhomogeneous part.

I find a discussion at stackflow(https://stackoverflow.com/questions/68916893/typeerror-numpy-dtypemeta-object-is-not-subscriptable). They saw the error is come from to the version of numpy. I try to install different version of numpy but failured.

I checked the data of v. Taking action as example, v is a list like [action1(20,8), action2(20,8),...,action8(8,)].

Do you know whether the data is coreect and have any suggestions on how to fix this? Thanks!

[MohitShridhar]

@ytx0323, was something modified in rlbench_utils.py? I can't find ln:269

[ytx0323]

Sorry, I added some print function to debug. The actual line should be 254.

[MohitShridhar]

Ah got it. Is this always an issue with the last observation? Something might have changed in robobase. If you comment out the add_final line, or add the observation before as the final, does the error go away?

(Sorry I don't have a personal GPU machine to debug this 😢)

[richielo]

Something might have changed in robobase

You can try reverting robobase to this commit

[dux2000]

Hi,

I have similar issue:

Error executing job with overrides: ['env=rlbench', 'env.dataset_root=/tmp/train_data_rnd_bg/', 'work_dir=/tmp/controller', 'demos=25', 'env.train_tasks=[take_lid_off_saucepan]', 'num_train_epochs=1000', 'action_sequence=20', 'batch_size=8', 'method.lr=1e-5', 'wandb.use=false', 'env.episode_length=2000'] Traceback (most recent call last): File "/home/-/genima/controller/train_act.py", line 292, in main workspace.train() File "/home/-/genima/controller/train_act.py", line 259, in train self._load_demos() File "/home/-/robobase/robobase/workspace.py", line 489, in _load_demos self.env_factory.load_demos_into_replay(self.cfg, self.replay_buffer) File "/home/-/genima/controller/env/rlbench.py", line 351, in load_demos_into_replay add_demo_to_replay_buffer(demo_env, buffer) File "/home/-/genima/controller/env/rlbench_utils.py", line 250, in add_demo_to_replay_buffer replay_buffer.add(obs, act, rew, term, trunc, **obs_and_info) File "/home/-/robobase/robobase/replay_buffer/uniform_replay_buffer.py", line 409, in add self._check_add_types(transition, self._storage_signature) File "/home/-/robobase/robobase/replay_buffer/uniform_replay_buffer.py", line 529, in _check_add_types raise ValueError( ValueError: arg action has shape (20, 8), expected (8,)

rlbench is up to date and robobase is at this commit https://github.com/robobase-org/robobase/commit/5e5705a56a05d8bb9cc7984c173c5fc4db84e82a

This is train_data folder for act training. Each folder has 57 images

I've even commented self._check_add_types to disable checks but then I get original error from ytx0323 -> ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (5,) + inhomogeneous part

If you comment out add_final as suggested you get this error: Unexpected error in training: list index out of range [2025-03-25 17:05:29,540][root][ERROR] - Traceback (most recent call last): File "/home/-/genima/controller/train_act.py", line 218, in _train self.agent.update( File "/home/-/genima/controller/method/genima_act.py", line 365, in update batch = next(replay_iter) File "/home/-/genima/controller/utils/dataloader.py", line 93, in next return self.sample(batch_size=len(batch_indices), indices=batch_indices) File "/home/-/robobase/robobase/replay_buffer/uniform_replay_buffer.py", line 842, in sample for k in samples[0].keys(): IndexError: list index out of range samples is basically empty []

[dux2000]

Hi,

I've managed to narrow down the problem to action_sequence. In the README file, action_sequence is set to 20, which is why we are getting the error:

ValueError: arg action has shape (20, 8), expected (8,)

If you set action_sequence to 1, everything works as expected. However, that means we're not fully taking advantage of ACT, are we?