🍄 Danish Fungi – Not Just Another Image Recognition Dataset

By Lukas Picek et al. MAIL

Description

Danish Fungi 2020 (DF20) is a new dataset for fine-grained visual categorization. The dataset is constructed from observations submitted to the Atlas of Danish Fungi and is unique in (i) its taxonomy-accurate labels with little to no label errors, (ii) highly unbalanced long-tailed class distribution, (iii) rich observation metadata about surrounding environment, location, time and device. DF20 has zero overlap with ImageNet, allowing unbiased comparison of models fine-tuned from publicly available ImageNet checkpoints. The proposed evaluation protocol enables testing the ability to improve classification using metadata – e.g. precise geographic location, habitat, and substrate, facilitates classifier calibration testing, and finally allows to study the impact of the device settings on the classification performance.

News

• 30.7. 2024: We made a new train/test split based on ObservationIDs where data from the same observation do not occur in the test set.
• 30.7. 2024: To distinguish it from the original split we call it DanishFungi24.
• 30.7. 2024: Updated baseline performance. All models are retrained and the results bellow updated (there is just a small drop in performance).
• 30.7. 2024: Model checkpoints are newly available at Hugging Face Hub Repository.

Data

To download the dataset files in CMD, use:

wget --no-check-certificate URL

In order to support research in fine-grained plant classification and to allow full reproducibility of our results, we share the training scripts and data tools.

• Checkpoints are available at Hugging Face Hub Repository.
• Train and Validation logs are available at Weights & Biases Workspace.

Installation

Python 3.10+ is required.

Local instalation

1. Install dependencies You can use any virtual or local environment. Just use the following commands in your terminal.

pip install -r requirements.txt

2. Login to Weights & Biases to log results [*optional].

wandb login

3. Login to Hugging Face Hub to save and download model checkpoints [*optional].

huggingface-cli login

Training

For training navigate to ./training folder. To run the training you can use the provided train.ipynb notebook or train.py CLI. In both you have to:

• Specify valid paths, wandb settings, etc. in train.ipynb or local environment and run. In the notebook all variables that must be "set" have "changethis" as value.

python train.py \
    --train-path $TRAIN_METADATA_PATH \
    --test-path $TEST_METADATA_PATH \
    --config-path ../configs/DF24M_224_config.yaml \
    --cuda-devices $CUDA_DEVICES \
    --wandb-entity $WANDB_ENTITY [**optional**] \
    --wandb-project $WANDB_PROJECT [**optional**] \
    --hfhub-owner $HFHUB_OWNER [**optional**]

os.environ["CUDA_VISIBLE_DEVICES"] = os.environ["CUDA_DEVICES"]

Exploiting metadata with late-fusion

To allow easy use of the available observation metadata (i.e., information about habitat, substrate etc.) we provide a notebook ./inference/metadata_fusion.ipynb that uses the late metadata-fusion (described in the paper Section 5.2).

Results

CNN Performance Evaluation [Updated]

Classification performance of selected CNN architectures on DF24 and DF24 - Mini. All networks share the settings described in the paper (Section 5.1) and were trained on 299×299 images.

ViT x CNN Performance Evaluation [Updated]

Classification results of selected CNN and ViT architectures on DF24 and DF24 - Mini dataset for two input resolutions 224×224, 384×384. Updated results based on new dataset split.

Input resolution: 224×224

Input resolution: 384×384

Metadata-fusion experiment [Updated]

Performance gains aquired by exploiting the observation metadata, i.e. Habitat (H), Substrate (S), and Month (M). Additionally, we provide performance gains based on the ObservationID grouping of calibrated predictions (average over class score). The method for late metadata-fusion is described in the paper (Section 5.2).

DF24 - ViT-Large/16 with image size 384×384. [Updated]

DF24 - ViT-Large/16 - 384×384 - With ObservationID grouping and calibration.

DF24 ViT-Base/16 with image size 224×224.

DF24 - ViT-Base/16 - 224×224 - With ObservationID grouping and calibration.

License

The code and dataset is released under the BSD License. There is some limitations for commercial usage. In other words, the training data, metadata, and models are available only for non-commercial research purposes only.

Citation

If you use Danish Fungi for your research or application, please consider citation:

@inproceedings{picek2022danish,
  title={Danish fungi 2020-not just another image recognition dataset},
  author={Picek, Luk{\'a}{\v{s}} and {\v{S}}ulc, Milan and Matas, Ji{\v{r}}{\'\i} and Jeppesen, Thomas S and Heilmann-Clausen, Jacob and L{\ae}ss{\o}e, Thomas and Fr{\o}slev, Tobias},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={1525--1535},
  year={2022}
}

@article{picek2022automatic,
  title={Automatic fungi recognition: deep learning meets mycology},
  author={Picek, Luk{\'a}{\v{s}} and {\v{S}}ulc, Milan and Matas, Ji{\v{r}}{\'\i} and Heilmann-Clausen, Jacob and Jeppesen, Thomas S and Lind, Emil},
  journal={Sensors},
  volume={22},
  number={2},
  pages={633},
  year={2022},
  publisher={Mdpi}
}

Contact

[Lukas Picek]([email protected], [email protected])