File "/home/mona/research/code/expose/expose/utils/plot_utils.py", line 831, in __call__ valid_mask = (color[3] > 0)[np.newaxis] IndexError: index 3 is out of bounds for axis 0 with size 3

[monacv]

two questions:

1. Is acceleratesupport - No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate' fine?
2. How do I resolve the following error and why is it happening?

(expose) mona@goku:~/research/code/expose$ python demo.py --image-folder ~/Downloads/sample1     --exp-cfg data/conf.yaml     --show=True     --output-folder ~/Downloads/sample_out     --save-params True     --save-vis True     --save-mesh True
INFO - 2021-02-04 16:06:35,899 - acceleratesupport - No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'
Processing with R-CNN: 100%|█████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.94s/it]
2021-02-04 16:06:41.101 | INFO     | __main__:main:241 - Saving results to: /home/mona/Downloads/sample_out
2021-02-04 16:06:41.105 | WARNING  | expose.models.attention.predictor:__init__:91 - Apply hand network on body: True
2021-02-04 16:06:41.105 | WARNING  | expose.models.attention.predictor:__init__:93 - Apply hand network on hands: True
2021-02-04 16:06:41.105 | WARNING  | expose.models.attention.predictor:__init__:95 - Predict hands: True
2021-02-04 16:06:41.105 | WARNING  | expose.models.attention.predictor:__init__:102 - Predict head: True
2021-02-04 16:06:41.105 | INFO     | expose.models.attention.predictor:__init__:109 - Condition hand on body: True
2021-02-04 16:06:41.106 | INFO     | expose.models.attention.predictor:__init__:112 - Condition hand wrist pose on body: True
2021-02-04 16:06:41.106 | INFO     | expose.models.attention.predictor:__init__:117 - Condition hand finger pose on body: True
2021-02-04 16:06:41.106 | INFO     | expose.models.attention.predictor:__init__:121 - Condition hand shape on body shape: False
2021-02-04 16:06:41.106 | INFO     | expose.models.attention.predictor:__init__:169 - Condition head on body: True
2021-02-04 16:06:41.106 | INFO     | expose.models.attention.predictor:__init__:170 - Condition expression on body: True
2021-02-04 16:06:41.106 | INFO     | expose.models.attention.predictor:__init__:172 - Condition shape on body: False
2021-02-04 16:06:41.106 | INFO     | expose.models.attention.predictor:__init__:173 - Condition neck pose on body: False
2021-02-04 16:06:41.106 | INFO     | expose.models.attention.predictor:__init__:175 - Condition jaw pose on body: True
2021-02-04 16:06:41.718 | INFO     | expose.models.attention.predictor:__init__:252 - Body model: SMPLXLayer(
  Gender: NEUTRAL
  Number of joints: 55
  Betas: 10
  Number of PCA components: 6
  Flat hand mean: False
  Number of Expression Coefficients: 10
  (vertex_joint_selector): VertexJointSelector()
)
2021-02-04 16:06:42.066 | INFO     | expose.models.backbone.hrnet:init_weights:487 - => init weights from normal distribution
2021-02-04 16:06:42.686 | WARNING  | expose.models.backbone.hrnet:load_weights:519 - => please download pre-trained models first!
2021-02-04 16:06:42.687 | WARNING  | expose.models.backbone.hrnet:load_weights:520 - data/network_weights/hrnet/imagenet/hrnet_w48-8ef0771d.pth does not exist!
2021-02-04 16:06:42.707 | INFO     | expose.models.common.networks:__init__:267 - Building iterative regressor with 3 stages
2021-02-04 16:06:42.707 | INFO     | expose.models.common.networks:__init__:282 - Detach mean: False
2021-02-04 16:06:42.839 | INFO     | expose.models.backbone.resnet:resnet18:113 - Loading pretrained ResNet-18
2021-02-04 16:06:42.909 | INFO     | expose.models.common.networks:__init__:267 - Building iterative regressor with 3 stages
2021-02-04 16:06:42.909 | INFO     | expose.models.common.networks:__init__:282 - Detach mean: False
2021-02-04 16:06:42.910 | INFO     | expose.models.attention.head_predictor:__init__:81 - Building head predictor with 3 stages
2021-02-04 16:06:43.062 | INFO     | expose.models.backbone.resnet:resnet18:113 - Loading pretrained ResNet-18
2021-02-04 16:06:43.133 | INFO     | expose.models.common.networks:__init__:267 - Building iterative regressor with 3 stages
2021-02-04 16:06:43.133 | INFO     | expose.models.common.networks:__init__:282 - Detach mean: False
2021-02-04 16:06:43.135 | INFO     | expose.models.attention.predictor:__init__:490 - 2D Head crop keyps loss: KeypointLoss(Norm type: L1)
2021-02-04 16:06:43.135 | INFO     | expose.models.attention.predictor:__init__:502 - 2D Left hand crop keyps loss: KeypointLoss(Norm type: L1)
2021-02-04 16:06:43.135 | INFO     | expose.models.attention.predictor:__init__:515 - 2D Left hand crop keyps loss: KeypointLoss(Norm type: L1)
2021-02-04 16:06:43.135 | INFO     | expose.models.common.smplx_loss_modules:__init__:48 - Stages to penalize: [-1]
2021-02-04 16:06:43.136 | INFO     | expose.models.common.smplx_loss_modules:__init__:400 - Stages to regularize: [-1]
2021-02-04 16:06:43.329 | INFO     | expose.utils.checkpointer:__init__:44 - Creating directory data/checkpoints
2021-02-04 16:06:43.329 | INFO     | expose.utils.checkpointer:load_checkpoint:90 - Load pretrained: False
2021-02-04 16:06:43.329 | WARNING  | expose.utils.checkpointer:load_checkpoint:93 - Loading checkpoint from data/checkpoints/model.ckpt!
2021-02-04 16:06:43.947 | WARNING  | expose.utils.checkpointer:load_checkpoint:121 - The following keys were not found: ['smplx.head_idxs', 'smplx.body_model.left_hand_components', 'smplx.body_model.right_hand_components', 'smplx.body_model.left_hand_mean', 'smplx.body_model.right_hand_mean', 'smplx.body_model.pose_mean', 'smplx.body_model.dynamic_lmk_bary_coords']
2021-02-04 16:06:43.947 | WARNING  | expose.utils.checkpointer:load_checkpoint:124 - The following keys were not expected: ['smplx.body_model.source_idxs', 'smplx.body_model.target_idxs', 'smplx.body_model.extra_joint_regressor', 'smplx.body_model.dynamic_lmk_b_coords', 'smplx.hand_predictor.hand_offset', 'smplx.hand_predictor.hand_model.extra_joints_idxs', 'smplx.hand_predictor.hand_model.faces_tensor', 'smplx.hand_predictor.hand_model.v_template', 'smplx.hand_predictor.hand_model.shapedirs', 'smplx.hand_predictor.hand_model.J_regressor', 'smplx.hand_predictor.hand_model.posedirs', 'smplx.hand_predictor.hand_model.parents', 'smplx.hand_predictor.hand_model.lbs_weights', 'smplx.hand_predictor.pca_decoder.pca_basis', 'smplx.hand_predictor.pca_decoder.inv_pca_basis', 'smplx.hand_predictor.pca_decoder.mean', 'smplx.head_predictor.head_offset', 'smplx.head_predictor.head_vertices_ids', 'smplx.head_predictor.head_model.faces_tensor', 'smplx.head_predictor.head_model.v_template', 'smplx.head_predictor.head_model.shapedirs', 'smplx.head_predictor.head_model.expr_dirs', 'smplx.head_predictor.head_model.J_regressor', 'smplx.head_predictor.head_model.posedirs', 'smplx.head_predictor.head_model.parents', 'smplx.head_predictor.head_model.lbs_weights', 'smplx.head_predictor.head_model.lmk_faces_idx', 'smplx.head_predictor.head_model.lmk_bary_coords', 'smplx.head_predictor.head_model.dynamic_lmk_faces_idx', 'smplx.head_predictor.head_model.dynamic_lmk_b_coords', 'smplx.head_predictor.head_model.neck_kin_chain', 'smplx.body_loss.edge_loss.gt_connections', 'smplx.body_loss.edge_loss.est_connections', 'smplx.hand_loss.edge_loss.gt_connections', 'smplx.hand_loss.edge_loss.est_connections', 'smplx.head_loss.edge_loss.gt_connections', 'smplx.head_loss.edge_loss.est_connections']
  0%|                                                                                                        | 0/2 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "demo.py", line 554, in <module>
    main(
  File "/home/mona/venv/expose/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
    return func(*args, **kwargs)
  File "demo.py", line 361, in main
    hd_orig_overlays = hd_renderer(
  File "/home/mona/venv/expose/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
    return func(*args, **kwargs)
  File "/home/mona/research/code/expose/expose/utils/plot_utils.py", line 831, in __call__
    valid_mask = (color[3] > 0)[np.newaxis]
IndexError: index 3 is out of bounds for axis 0 with size 3

[foermozhu]

I also encountered this problem. Have you solved it now？

[booker-max]

Do you have the training code for this paper? Can you share it with me? This is my email address [email protected]，Thank you very much.！

[biansy000]

@monacv @foermozhu I encounter the problem too. The bug occurs because in some modes, pyrender produces images with RGB format instead of RGBA format as intended. You can just manually add the last 'A' dimension to the image (but this may cause the finally rendered picture look a little different from intended).

For me, I add the following codes at line 831

if len(color) < 4:
    s = list(color.shape)
    s[0] = 1
    new_axis_arr = np.ones(s)
    color = np.concatenate([color, new_axis_arr], axis=0)

[Junyi-99]

Hi, I encounter the problem too.

[Junyi-99]

@monacv @foermozhu I encounter the problem too. The bug occurs because in some modes, pyrender produces images with RGB format instead of RGBA format as intended. You can just manually add the last 'A' dimension to the image (but this may cause the finally rendered picture look a little different from intended).

For me, I add the following codes at line 831

if len(color) < 4:
    s = list(color.shape)
    s[0] = 1
    new_axis_arr = np.ones(s)
    color = np.concatenate([color, new_axis_arr], axis=0)

I edited expose/expose/utils/plot_utils.py at line 831 with your code, and it fixed for me, thanks a lot!