[Bug] Leaderboard submission failed

[henryzhengr]

Prerequisite

• [X] I have searched Issues and Discussions but cannot get the expected help.
• [X] I have read the FAQ documentation but cannot get the expected help.
• [X] The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

NA

Reproduces the problem - code sample

NA

Reproduces the problem - command or script

NA

Reproduces the problem - error message

I tried to submit to the leaderboard but the server doesnt seem to work. I even tried resubmitting files that previously succeeded, but it also failed this time.

Additional information

No response

[henryzhengr]

Hi @mxh1999 @Tai-Wang Moreover, I would like to ask if the results in the paper refer to the private split used by the test server, or the publicly available validation set?

[mxh1999]

Hi @henryzhengr , I apologize for the late reply. Huggingface has updated their apis recently and we were adapting it accordingly. The leaderboard is ready for submission now. If you have any questions, please feel free to ask here!

[Tai-Wang]

We will clarify the benchmark settings and update the results in the paper when polishing the benchmarks next. For now, we show the benchmark results on the validation set in the GitHub repo and results on the test set on the Huggingface leaderboard. We will integrate EmbodiedScan and MMScan benchmarks with an updated approach in the next two months.