sleap icon indicating copy to clipboard operation
sleap copied to clipboard

Improve DLC import integration

Open talmo opened this issue 3 years ago • 0 comments

Currently we can import DLC labeling projects for single or multi-animal, in old and new formats, but we're still missing a few pieces of metadata that may be helpful for complete integration:

  1. Training set splits: We store train/val/test split metadata in TrainingJobConfig, not the Labels, so this would need to get imported separately. It is also not clear where it's stored in the DLC outputs?
  2. Cropping: We do not currently support ROI cropping (#721). We should add this functionality once it's supported in SLEAP.
  3. Link single images back to original videos: DLC exports training data as folders of images corresponding to labeled frames, while we just embed them in SLEAP .pkg.slp files. When importing labeled folders, it would be great to be able to link back to the original media files (.mp4, .avi, etc) by parsing DLC metadata, possibly requiring access to other files not included in the exported training data.
  4. Skeleton edges: We currently only import skeleton nodes, but not the edges from DLC projects. This was irrelevant for single-animal videos since they were just used for visualization, but matter more now that we support importing multi-animal datasets.

Relevant issues: #606

This issue might get broken down into multiple sub-tasks as we explore the feasibility of each.

talmo avatar May 12 '22 22:05 talmo