Awesome Deep Ecology

A curated list of deep learning resources for computer vision, inspired by awesome-deep-vision and awesome-computer-vision.

We are looking for additional maintainers! Let me know (pcg19 at duke) if interested.

An introduction to Deep Learning for Ecology with some neural networks you can run in the cloud to get a more intuitive understanding of deep learning, as well as suggestions for papers, videos, and quick resources, can be found here.

Papers

Reviews

• Applications for deep learning in ecology
  • Sylvain Christin, Eric Hervet, Nicolas Lecomte
  • https://www.biorxiv.org/content/early/2018/05/30/334854
• A computer vision for animal ecology
  • Ben G. Weinstein
  • https://besjournals.onlinelibrary.wiley.com/doi/10.1111/1365-2656.12780
• Machine learning for image based species identification
  • Jana Wäldchen
  • https://besjournals.onlinelibrary.wiley.com/doi/pdf/10.1111/2041-210X.13075

Detection

• Deep learning for benthic fauna identification. OCEANS 2016 MTS/IEEE Monterey, OCE 2016.
  • Marburg, A., & Bigham, K. (2016).
  • doi:10.1109/OCEANS.2016.7761146
• Right whale recognition using convolutional neural networks.
  • Polzounov, A., Terpugova, I., Skiparis, D., & Mihai, A. (2016).
  • http://arxiv.org/abs/1604.05605

Camera Traps

• Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning
  • Mohammed Sadegh Norouzzadeh, Anh Nguyen, Margaret Kosmala, Ali Swanson, Meredith Palmer, Craig Packer, Jeff Clune
  • https://arxiv.org/abs/1703.05830 or http://www.pnas.org/content/115/25/E5716
• Machine learning to classify animal species in camera trap images: applications in ecology
  • Tabak et al. https://www.biorxiv.org/content/early/2018/06/14/346809
• Recognition in Terra Incognita
  • Sara Beery, Grant Van Horn, and Pietro Perona
  • https://arxiv.org/pdf/1807.04975.pdf
  • Dataset: https://beerys.github.io/CaltechCameraTraps/
• Deep Learning Object Detection Methods for Ecological Camera Trap Data
  • Stefan Schneider, Graham W. Taylor, Stefan C. Kremer
  • https://arxiv.org/abs/1803.10842
• A comparison of visual features used by humans and machines to classify wildlife
  • https://www.biorxiv.org/content/early/2018/10/22/450189.full.pdf+html
• Scene‐specific convolutional neural networks for video‐based biodiversity detection
  • Ben G. Weinstein
  • https://besjournals.onlinelibrary.wiley.com/doi/abs/10.1111/2041-210X.13011
• Identifying animal species in camera trap images using deep learning and citizen science
  • Marco Willi, Ross T. Pitman, Anabelle W. Cardoso, Christina Locke, Alexandra Swanson, Amy Boyer, Marten Veldthuis, Lucy Fortson
  • https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.13099

Aerial Imagery

• Multi-modal survey of Adélie penguin mega-colonies reveals the Danger Islands as a seabird hotspot
  • Alex Borowicz, Philip McDowall, Casey Youngflesh, Thomas Sayre-McCord, Gemma Clucas, Rachael Herman, Steven Forrest, Melissa Rider, Mathew Schwaller, Tom Hart, Stéphanie Jenouvrier, Michael J. Polito, Hanumant Singh & Heather J. Lynch
  • https://www.nature.com/articles/s41598-018-22313-w
• Detecting Wildlife in Unmanned Aerial Systems Imagery Using Convolutional Neural Networks Trained with an Automated Feedback Loop
  • Bowley, C., Mattingly, M., Barnas, A., Ellis-Felege, S., & Desell, T. (2018).
  • doi:10.1007/978-3-319-93698-7_6
• A comparison of deep learning and citizen science techniques for counting wildlife in aerial survey images
  • Colin J. Torney David J. Lloyd‐Jones Mark Chevallier David C. Moyer Honori T. Maliti Machoke Mwita Edward M. Kohi Grant C. Hopcraft. https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.13165

Audio

• Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challenge. Stowell et al. 2018
• Oisin Mac Aodha, Rory Gibb, Kate E. Barlow, Ella Browning, Michael Firman, Robin Freeman, Briana Harder, Libby Kinsey, Gary R. Mead, Stuart E. Newson, Ivan Pandourski, Stuart Parsons, Jon Russ, Abigel Szodoray-Paradi, Farkas Szodoray-Paradi, Elena Tilova, Mark Girolami, Gabriel Brostow, Kate E. Jones
  • http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005995
• Report: Spotted Owl Acoustic Monitoring

Other

• Fast accurate fish detection and recognition of underwater images with Fast R-CNN
  • Xiu Li, Min Shang, Hongwei Qin, Liansheng Chen
  • https://ieeexplore.ieee.org/document/7404464/
• Bat detective—Deep learning tools for bat acoustic signal detection

Tracking

• Automated Analysis of Marine Video With Limited Data
  • Deborah Levy, Yuval Belfer, Elad Osherov, Eyal Bigal, Aviad P. Scheinin, Hagai Nativ, Dan Tchernov, and Tali Treibitz
  • http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w28/Levy_Automated_Analysis_of_CVPR_2018_paper.pdf

Semantic Segmentation

Pose Estimation

• Omni-supervised joint detection and pose estimation for wild animals
  • Zheng et al. 2018.
  • https://www.sciencedirect.com/science/article/abs/pii/S0167865518308742
• Fast animal pose estimation using deep neural networks
  • Talmo D. Pereira, Diego E. Aldarondo, Lindsay Willmore, Mikhail Kislin, Samuel S.-H. Wang, Mala Murthy, Joshua W. Shaevitz
  • https://www.biorxiv.org/content/early/2018/05/30/331181
  • Code: https://github.com/talmo/leap

Other Topics

Software

• Annotation Tools

  • VGG Image Annotator (VIA)
    • http://www.robots.ox.ac.uk/~vgg/software/via/
  • Visual Object Tagging Tool (VoTT)
    • https://github.com/Microsoft/VoTT

• Well maintained implementations of neural networks

  • Mask RCNN
    • https://github.com/matterport/Mask_RCNN
  • RetinaNet
    • https://github.com/fizyr/keras-retinanet
  • TensorFlow Object Detection API (Faster RCNN, Mask RCNN, SSD, Region-based Fully Convolutional Networks R-FCN)
    • https://github.com/tensorflow/models/tree/master/research/object_detection

• Analysis tools

  • Animal Scanner for Camera traps: https://onlinelibrary.wiley.com/doi/full/10.1002/ece3.4747
  • Indoor Tracking: Romero-Ferrero, F., Bergomi, M. G., Hinz, R., Heras, F. J. H., & De Polavieja, G. G. (2018). idtracker.ai: Tracking all individuals in large collectives of unmarked animals, (Protocol 1).
  • doi:10.1101/280735
  • Motion Detection: http://benweinstein.weebly.com/deepmeerkat.html
  • Animal Detection Network: semi-automated annotation tools for camera traps
  • Project Zamba: A Python package for identifying 23 kinds of animals in camera trap videos
  • Machine Learning for Image Classification R Package: https://github.com/mikeyEcology/MLWIC/

• Models with open-source training code

  • Norouzzadeh et al. Snapshot Serengeti model training
  • Willi et al. Camera-Trap-Classifier model training

Datasets

• LILA BC: a repository for labeled conservation data with ~20M images, including a list of other data sets
• CaltechCameraTraps: 243,187 annotated images from 140 camera locations in the Southwest US
• Snapshot Serengeti: 1.2M annotated sequences from East Africa
• eMammal/University of Missouri Camera Traps: 11 GB of annotated images from Yousif et al. 2018
• NIPS4Bplus: a richly annotated birdsong audio dataset
• Camera-Trap datasets used in Willi et. al: over 7M annotated images from South Africa, Tanzania, and Wisconsin

Further Reading

• Survey of machine learning applications for camera traps

Contributing

Please feel free to pull requests to add papers.

Sharing

• Share on Twitter
• Share on Facebook
• Share on Google Plus
• Share on LinkedIn