parameter initialize error for clip model

[dljzx]

Thanks for your code. While Implementing clip rl Model by changing TRANSFER_TYPE to "clip_vit_b32", code faced an error like this: Traceback (most recent call last): File "/nvme/.../anaconda3/envs/instructrl/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/nvme/.../anaconda3/envs/instructrl/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/nvme/.../workspace/robot/instructrl/instructrl/instructrl_main.py", line 546, in app.run(main) File "/nvme/.../anaconda3/envs/instructrl/lib/python3.9/site-packages/absl/app.py", line 312, in run _run_main(main, args) File "/nvme/.../anaconda3/envs/instructrl/lib/python3.9/site-packages/absl/app.py", line 258, in _run_main sys.exit(main(argv)) File "/nvme/.../workspace/robot/instructrl/instructrl/instructrl_main.py", line 322, in main params = init(next_rng(model.rng_keys()), dummy_input, deterministic=False)[ File "/nvme/.../workspace/robot/instructrl/instructrl/instructrl_main.py", line 319, in init return model.init(*args, **kwargs) File "/nvme/.../workspace/robot/instructrl/instructrl/model.py", line 292, in call num_obs_token, image_embed, action_embed, state_embed = self.encode(batch) File "/nvme/.../workspace/robot/instructrl/instructrl/model.py", line 453, in encode text_emb = self.pt_model.apply( File "/nvme/.../workspace/robot/instructrl/instructrl/models/openai/layers.py", line 456, in encode_text x = self.text(text) File "/nvme/.../workspace/robot/instructrl/instructrl/models/openai/layers.py", line 375, in call positional_embedding = self.param( flax.errors.ScopeParamShapeError: Inconsistent shapes between value and initializer for parameter "positional_embedding" in "/text": (77, 512), (4, 512). (https://flax.readthedocs.io/en/latest/flax.errors.html#flax.errors.ScopeParamShapeError)

How can i solve it?

[jackandsorrel]

i had this error as well. how did you solve it?