qasfb
qasfb
Hey ! Can you say a bit more about your setup, how many GPUs, what batch size and what config in general ?
How did you pick the hyperparameters in this config ? I see layerwise decay 0.9 and momentum teacher 0.9995, and i'm pretty sure these were not in DINO. Similarly the...
To disable: layerwise decay = 1.0 Otherwise I don't know, I have not tried to reproduce DINO with this codebase so I don't know for sure if it would work...
So in dinov2 we can iterate over the dataset forever, and training will stop when a given number of iterations is reached. It is likely that your dataset stops iterating...
Does your data_loader class stop providing batches after the first pass over the dataset ? if yes, this is the problem. Dinov2 requires your data_loader to keep generating batches of...
The model does the resizing automatically during inference I think, no ?
if you want to onnx-export for inference at a fixed resolution, i think you can interpolate the position embeddings in advance, with antialias, and not do it at run-time
It's just a logging issue, it displays the batch size per gpu; maybe we can put a better name
I depends on your inter-node connectivity
if infiniband is not used, maybe there is a problem with the cluster configuration ? are you able to run nccl-tests, and does it give the perf that it should...