lightly-ai
Add correctness check before uploading samples and embeddings
When uploading a dataset using lightly-upload where the embeddings are not in the right csv format the CLI should break before uploading part of the data.
E.g. when one of the columns has the wrong name:
Furthermore, we should make sure that we don't upload embedding files that don't match our specs.
We already make sure that we don't upload embedding files not matching the specs, as they cause the reordering according to the mapping to fail (see below). However, you are right, we should check the correctness before doing the upload of samples. This would prevent errors if the upload of the dataset and embeddings is triggered at once, but not if first the dataset and then the embeddings are uploaded. https://github.com/lightly-ai/lightly/blob/e6565c0cd58e17472841a16943c360266de212a2/lightly/api/api_workflow_upload_embeddings.py#L72-L85
