pliang279
Question about the MUJOCO PUSH accuracy
Thanks for your excellent work! I have questions about the results in Table 17. Is there a scientific notation unit missed since my reproduction results are about 0.0023 in MSE?
Just like in Table 18, where the scientific notation is provided:
Thanks for your time. Here are my test logs: Training Time: 616.6786675453186 Training Peak Mem: 22735.2109375 Training Params: 1904986 mse: 0.002326280577108264 Inference Time: 2.2704453468322754 Inference Params: 1904986 Testing on noisy data (image)... 10%|█ | 1/10 [00:02<00:19, 2.18s/it]mse: 0.002326280577108264 mse: 1.2753043174743652 30%|███ | 3/10 [00:06<00:15, 2.21s/it]mse: 1.7230396270751953 mse: 2.0257174968719482 50%|█████ | 5/10 [00:11<00:11, 2.22s/it]mse: 2.305746078491211 60%|██████ | 6/10 [00:13<00:08, 2.22s/it]mse: 2.6151041984558105 70%|███████ | 7/10 [00:15<00:06, 2.22s/it]mse: 2.7902309894561768 mse: 2.8953778743743896 90%|█████████ | 9/10 [00:19<00:02, 2.23s/it]mse: 2.9822394847869873 mse: 3.0394344329833984 relative robustness (image, MSE): 0.0 effective robustness (image, MSE): 0.0 100%|██████████| 10/10 [00:22<00:00, 2.22s/it] 0%| | 0/10 [00:00<?, ?it/s]Plot saved as My method-gentle push image-image-MSE Testing on noisy data (proprio)... mse: 0.002326280577108264 20%|██ | 2/10 [00:04<00:19, 2.40s/it]mse: 0.07984583079814911 30%|███ | 3/10 [00:07<00:16, 2.39s/it]mse: 0.13955439627170563 40%|████ | 4/10 [00:09<00:14, 2.39s/it]mse: 0.20418862998485565 50%|█████ | 5/10 [00:11<00:11, 2.38s/it]mse: 0.26611292362213135 60%|██████ | 6/10 [00:14<00:09, 2.38s/it]mse: 0.32232439517974854 70%|███████ | 7/10 [00:16<00:07, 2.38s/it]mse: 0.3833681344985962 mse: 0.4626495838165283 90%|█████████ | 9/10 [00:21<00:02, 2.38s/it]mse: 0.5830950736999512 100%|██████████| 10/10 [00:23<00:00, 2.38s/it] mse: 0.7971442341804504 relative robustness (proprio, MSE): 0.0 effective robustness (proprio, MSE): 0.0 0%| | 0/10 [00:00<?, ?it/s]Plot saved as My method-gentle push proprio-proprio-MSE Testing on noisy data (haptics)... 10%|█ | 1/10 [00:02<00:21, 2.37s/it]mse: 0.002326280577108264 mse: 0.0036232746206223965 30%|███ | 3/10 [00:07<00:16, 2.38s/it]mse: 0.0056821503676474094 40%|████ | 4/10 [00:09<00:14, 2.37s/it]mse: 0.0069841258227825165 50%|█████ | 5/10 [00:11<00:11, 2.37s/it]mse: 0.007524526212364435 60%|██████ | 6/10 [00:14<00:09, 2.37s/it]mse: 0.007330612279474735 70%|███████ | 7/10 [00:16<00:07, 2.38s/it]mse: 0.006867253687232733 mse: 0.006039843894541264 90%|█████████ | 9/10 [00:21<00:02, 2.37s/it]mse: 0.005186266731470823 100%|██████████| 10/10 [00:23<00:00, 2.38s/it] mse: 0.0038046431727707386 relative robustness (haptics, MSE): 0.0 effective robustness (haptics, MSE): 0.0 0%| | 0/10 [00:00<?, ?it/s]Plot saved as My method-gentle push haptics-haptics-MSE Testing on noisy data (controls)... mse: 0.002326280577108264 20%|██ | 2/10 [00:04<00:19, 2.38s/it]mse: 0.06037278473377228 mse: 0.056742727756500244 40%|████ | 4/10 [00:09<00:14, 2.38s/it]mse: 0.0527164526283741 mse: 0.04934581369161606 60%|██████ | 6/10 [00:14<00:09, 2.38s/it]mse: 0.046973370015621185 70%|███████ | 7/10 [00:16<00:07, 2.37s/it]mse: 0.04735881835222244 mse: 0.04990621283650398 90%|█████████ | 9/10 [00:21<00:02, 2.38s/it]mse: 0.060220394283533096 mse: 0.0837954431772232 relative robustness (controls, MSE): 0.0 effective robustness (controls, MSE): 0.0 100%|██████████| 10/10 [00:23<00:00, 2.38s/it] 0%| | 0/10 [00:00<?, ?it/s]Plot saved as My method-gentle push controls-controls-MSE Testing on noisy data (multimodal)... mse: 0.002326280577108264 20%|██ | 2/10 [00:04<00:18, 2.37s/it]mse: 0.40776482224464417 mse: 0.629368782043457 40%|████ | 4/10 [00:09<00:14, 2.38s/it]mse: 0.7287225127220154 mse: 0.7674158811569214 60%|██████ | 6/10 [00:14<00:09, 2.37s/it]mse: 0.8007391691207886 mse: 0.8004142045974731 80%|████████ | 8/10 [00:18<00:04, 2.37s/it]mse: 0.8466270565986633 90%|█████████ | 9/10 [00:21<00:02, 2.37s/it]mse: 0.9301233291625977 100%|██████████| 10/10 [00:23<00:00, 2.37s/it] mse: 1.0777761936187744 relative robustness (multimodal, MSE): 0.0 effective robustness (multimodal, MSE): 0.0 Plot saved as My method-gentle push multimodal-multimodal-MSE
