Train model with data of no labels?

[ambipomyan]

Hi, I have a question about how to handle the dataset and label.txt when I want to train the model. I am a bit confused when I see the settings for labels since the training is of unsupervised learning, then how I handle the data structure of the dataset and contexts of label.txt for training give that I have only data with no label?

I guess the labels should not do anything for the unsupervised training and now I just put the inputs, which are images with no label, into separated folders and recognize the name of the folders as the name of class in order to fit the format of dinov2 inputs. In this way, the name of class and the images in the folders are definitely not matching. Will this work for training?

Thank you in advance!

dataset

- ROOT
   |-- train
   |   |-- folder0
   |   |    |-- folder0_01.jpeg
   |   |    `-- ...
   |   |-- folder1
   |   |    |-- folder1_01.jpeg
   |   |    `-- ...
   |   `-- ...
   |-- val
   |-- test
   `-- label.txt

label.txt

folder0, folder0
folder1, folder1
...

[yyyyyyfs]

As far as I know，training or finetuning this model is not self-supervised. It is also need label，so if you want to train your dataset， you should generate your labels.txt, and modify the data_loader to adapt your dataset.

[ddstone]

As far as I know，training or finetuning this model is not self-supervised. It is also need label，so if you want to train your dataset， you should generate your labels.txt, and modify the data_loader to adapt your dataset.

After i checked the source code, i.e., the training code in ssl_meta_arch.py, what i found is the training process did not use the label but only images, which is the definition of self-supervised learning. I'm not sure the purpose of the label of dataset , but i think there is a way to leverage the project to pretrain on custom dataset with no label.

[TheoMoutakanni]

Hello, The training of DINOv2 is fully unsupervised. We don't use labels during the training but we use them during the validation process where we fit a linear layer on top of our frozen features to check that the model is training well. You shouldn't need labels to launch a training of DINOv2, our datasets have labels for evaluation but if you don't have labels you can return something like '0' or 'np.nan'.

Look at this file to see what you need for a minimal dataset: https://github.com/facebookresearch/dinov2/blob/main/dinov2/data/datasets/extended.py

To make a new dataset without labels, you would have to make a class that inherits ExtendedVisionDataset and create functions for get_image_data (should return an image as an array), get_target (in your case this should return 0 or nan) and __len__ (this should returns the lenght of the dataset).

You can take inspiration from the ImageNet dataset class in https://github.com/facebookresearch/dinov2/blob/main/dinov2/data/datasets/image_net.py

But keep in mind that you don't need to copy all functions in the ImageNet dataset.

[ddstone]

Hello, The training of DINOv2 is fully unsupervised. We don't use labels during the training but we use them during the validation process where we fit a linear layer on top of our frozen features to check that the model is training well. You shouldn't need labels to launch a training of DINOv2, our datasets have labels for evaluation but if you don't have labels you can return something like '0' or 'np.nan'.

Look at this file to see what you need for a minimal dataset: https://github.com/facebookresearch/dinov2/blob/main/dinov2/data/datasets/extended.py

To make a new dataset without labels, you would have to make a class that inherits ExtendedVisionDataset and create functions for get_image_data (should return an image as an array), get_target (in your case this should return 0 or nan) and __len__ (this should returns the lenght of the dataset).

You can take inspiration from the ImageNet dataset class in https://github.com/facebookresearch/dinov2/blob/main/dinov2/data/datasets/image_net.py

But keep in mind that you don't need to copy all functions in the ImageNet dataset.

Appreciate your rely, i've launched the training with dataset without labels, by mocking the image_net.py. Thank you!

[HDL-YD]

Hello, The training of DINOv2 is fully unsupervised. We don't use labels during the training but we use them during the validation process where we fit a linear layer on top of our frozen features to check that the model is training well. You shouldn't need labels to launch a training of DINOv2, our datasets have labels for evaluation but if you don't have labels you can return something like '0' or 'np.nan'. Look at this file to see what you need for a minimal dataset: https://github.com/facebookresearch/dinov2/blob/main/dinov2/data/datasets/extended.py To make a new dataset without labels, you would have to make a class that inherits ExtendedVisionDataset and create functions for get_image_data (should return an image as an array), get_target (in your case this should return 0 or nan) and __len__ (this should returns the lenght of the dataset). You can take inspiration from the ImageNet dataset class in https://github.com/facebookresearch/dinov2/blob/main/dinov2/data/datasets/image_net.py But keep in mind that you don't need to copy all functions in the ImageNet dataset.

Appreciate your rely, i've launched the training with dataset without labels, by mocking the image_net.py. Thank you!

I've also recently trained a backbone network with my own dataset. Can you share your dataset structure and the changed loading dataset code?

[csaroff]

I have a fork that trains a dinov2 model with an arbitrary unlabeled dataset using skypilot.

https://github.com/csaroff/dinov2/tree/main/sky

Should be easy enough to edit the config to pull down your dataset instead of mine

Only caveat is that resuming training from a checkpoint isn't working(re-running training will overwrite old checkpoints instead of resuming) so you either can't use spot instances, or you'll have to debug that particular issue.

[shiyongde]

same problem

[ayushnangia]

did you figure out how to just pretrain by initializing the weights from the model weights given? @csaroff @shiyongde