Access to Test Dataset Ground Truth for Benchmarking

[akashghimireOfficial]

Hi,

Thank you for sharing this excellent work, including the dataset and benchmark code! I noticed that the ground truth labels are only provided for the training and validation sets.

How can we evaluate our models on the test set? The paper mentions benchmarking on the test set, so I assume ground truth exists. Could you provide guidance or access to the test set labels for evaluation? It’s crucial for my class project.

Thanks!

[JaWeyl]

Hi @akashghimireOfficial,

Please note that the test set of the PhenoBench dataset is hidden, i.e., we provide a competition on CodaLab where you can upload your predictions of the test set and we perform a server-sided evaluation. You can find a list of all competitions here.

[chenfh21]

Hi @JaWeyl , I have tried uploading my test files multiple times but was unsuccessful. I don't know where the problem lies.

[jbehley]

Just some general questions that we can investigate what is going wrong:

1. to which codalab competition did you submit; can you provide the URL (e.g., https://codalab.lisn.upsaclay.fr/competitions/13654)
2. What is your username on codalab.
3. Please have a look at the "Evaluation" page on the Codalab competitions, where we specify the folder structure that a zip file should have (e.g., https://codalab.lisn.upsaclay.fr/competitions/13654#learn_the_details-evaluation). Note that this is different for different competitions. There should be no subfolders in the zip file.
4. We provide a validator script that checks for the consistency of the zip file; please see the instructions: https://github.com/PRBonn/phenobench?tab=readme-ov-file#codalab-submission-validator-phenobench-validator Let us know where in the process it get's stuck.