Towards DeepLabCut 3.0: SuperAnimal Inference

[MMathisLab]

In 2024 we are moving to PyTorch (fully)🔥 ! This has been on the roadmap for some years, and in 2023 the team worked to completely overall the core of deeplabcut with the generous support from CZI EOSS Program. It will feel (mostly) seamless to users to move to this version, and it will have the same high-level API.

This PR introduces (only) the model inference (video_analysis, etc.) for DLC3.0 pytorch trained models.

• The first models that we release are our new foundation animal pose models, which are documented in Ye et al. on arXiv.
• The Model Cards and details can be found here and here.

-There are a huge number of people behind this, so check out the contributors who have really worked incredibly hard to build a well engineered and tested code base!

• Lastly, better API documentation is forthcoming 🥳, but you can check out the code in the meantime.

[MMathisLab]

@n-poulsen please check codespell; i.e, exclusion of some files is needed, and two clear fixes needed :D :

Error: ./deeplabcut/pose_estimation_pytorch/config/make_pose_config.py:108: BU ==> BY, BE, BUT, BUG, BUN, BUD, BUY, BUM
Error: ./deeplabcut/pose_estimation_pytorch/config/make_pose_config.py:210: bu ==> by, be, but, bug, bun, bud, buy, bum
Error: ./deeplabcut/pose_estimation_pytorch/config/base/base.yaml:34: bu ==> by, be, but, bug, bun, bud, buy, bum
Error: ./deeplabcut/pose_estimation_pytorch/apis/utils.py:470: bu ==> by, be, but, bug, bun, bud, buy, bum
Error: ./deeplabcut/pose_estimation_pytorch/apis/analyze_videos.py:235: BU ==> BY, BE, BUT, BUG, BUN, BUD, BUY, BUM
Error: ./deeplabcut/pose_estimation_pytorch/modelzoo/inference.py:108: BU ==> BY, BE, BUT, BUG, BUN, BUD, BUY, BUM
Error: ./deeplabcut/pose_estimation_pytorch/data/postprocessor.py:289: BU ==> BY, BE, BUT, BUG, BUN, BUD, BUY, BUM
Error: ./deeplabcut/pose_estimation_pytorch/data/transforms.py:[17](https://github.com/DeepLabCut/DeepLabCut/actions/runs/7360532599/job/20036759485?pr=2472#step:4:18)3: overlayed ==> overlaid
Error: ./deeplabcut/pose_estimation_pytorch/runners/base.py:28: BU ==> BY, BE, BUT, BUG, BUN, BUD, BUY, BUM
Error: ./deeplabcut/pose_estimation_pytorch/runners/schedulers.py:34: beggining ==> beginning

[MMathisLab]

@nastya236 and @n-poulsen tests are failing due to missing import (I think), so please double check, thanks!

==================================== ERRORS ====================================
______________ ERROR collecting tests/test_predict_supermodel.py _______________
ImportError while importing test module '/home/runner/work/DeepLabCut/DeepLabCut/tests/test_predict_supermodel.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_predict_supermodel.py:13: in <module>
    from deeplabcut.modelzoo.api import superanimal_inference
E   ModuleNotFoundError: No module named 'deeplabcut.modelzoo.api'

[nastya236]

I forgot to update tensorflow test file for superanimal inference. @MMathisLab now is fixed here: PR#2473.

[MMathisLab]

A few more tests need checked like for dekr:

FAILED tests/test_dekr_predictor.py::test_DEKRPredictor - Failed: DEKRPredictor forward pass raised an exception: tuple indices must be integers or slices, not str

[n-poulsen]

I've fixed the file headers, but only for files in the deeplabcut/modelzoo and deeplabcut/pose_estimation_pytorch folders. I believe most headers need to be updated (switching from master to main as the default branch), but this should be done in a separate PR. The following was also done:

7425cc8

• fixed typos
• removed duplicate file headers
• updated the codespell configuration to ignore bu, BU, td, TD

9d5e2dd

• added a docstring for parse_project_model_name in deeplabcut/modelzoo/utils.py
• removed the failing test tests/test_dekr_predictor.py (a legacy test that I didn't catch when preparing the branch)
• improved the description of the TODO for the TokenPose base configuration

Some checks still seems to be failing, but that's due to an incorrect hash when installing FFmpeg on OSX, which I don't believe is an issue from our end.

[MMathisLab]

This PR/branch will stay for historical purposes but is going to be superseded by a new, fuller integration, PR ASAP.