scripts/predict_new_vids.py should save loss metrics as csv

[EugenioBertolini]

Hello,

Thank you so much for this amazing module/library. It's really well written and it has a very informative documentation.

(This issue should be more like a pull request, but I'm quite new on github and I don't have too much time right now. Sorryyy)

State

I successfully used train_hydra.py to train the model on my dataset.

Problem

I now would like to use predict_new_vids.py to do inference on some new videos coming from an additional dataset. I would to use the pca_singleview_error and the temporal_norm to select some frames from these videos to manually relabel. The problem is that predict_new_vids.py as it's written, outputs only the likelihood, which is not very informative.

Solution

I modified this predict_new_vids.py so that it outputs the estimated keypoints, the pca_singleview_error, and the temporal_norm in 3 separate csv files as train_hydra.py does during its "predict" phase.

from lightning_pose.utils.scripts import (
    compute_metrics,
    export_predictions_and_labeled_video,
    get_data_module,
    get_dataset,
    get_imgaug_transform,
)

...

@typechecked
class VideoPredPathHandler:

    # ...

    def build_pred_file_basename(self, extra_str="") -> str:
        # return "%s_%s%s%s.csv" % (
        #     self.video_basename,
        #     self.model_cfg.model.model_type,
        #     self.loss_str,
        #     extra_str,
        # )
        return f"{self.video_basename}.csv"

...

@hydra.main(config_path="configs", config_name="config_mirror-mouse-example")
def predict_videos_in_dir(cfg: DictConfig):

    # ...

    for _, hydra_relative_path in enumerate(cfg.eval.hydra_paths):

        # ...

        for video_file in video_files:

            # ...

            print(f"\n\n{prediction_csv_file = }\n\n")

            export_predictions_and_labeled_video(
                video_file=video_file,
                cfg=cfg,
                ckpt_file=ckpt_file,
                prediction_csv_file=prediction_csv_file,
                labeled_mp4_file=labeled_mp4_file,
                trainer=trainer,
                model=model,
                data_module=data_module,
                save_heatmaps=cfg.eval.get(
                    "predict_vids_after_training_save_heatmaps", False
                ),
            )
            # compute and save various metrics
            try:
                compute_metrics(
                    cfg=cfg,
                    preds_file=prediction_csv_file,
                    data_module=data_module,
                )
            except Exception as e:
                print(f"Error predicting on video {video_file}:\n{e}")
                continue

if __name__ == "__main__":
    predict_videos_in_dir()

[themattinthehatt]

Hi @EugenioBertolini, thanks for the kind words! Glad you were able to get the training up and running without issues :)

It would be great if you could add a pull request for this - it shows more people are engaged with the package development, and gives you credit for your contributions! Let me know if you like to give it a try, here are some instructions: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork.

[EugenioBertolini]

Ok, as soon as I have some spare time today or tomorrow I'll give it a try

[themattinthehatt]

Awesome, thank you so much! No rush

On Tue, Mar 5, 2024, 09:39 Eugenio Bertolini @.***> wrote:

Ok, as soon as I have some spare time today or tomorrow I'll give it a try

— Reply to this email directly, view it on GitHub https://github.com/danbider/lightning-pose/issues/131#issuecomment-1978335689, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADJPWM4HZG5SIGSXECALSL3YWWHFXAVCNFSM6AAAAABEGXC27OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYGMZTKNRYHE . You are receiving this because you were assigned.Message ID: @.***>

[themattinthehatt]

Hi @EugenioBertolini, just checking in - is this still a PR you'd like to make?

[EugenioBertolini]

Yeah, sorry. I've been super busy these 2 weeks. I couldn't find any moment to get myself learn how to do a pull request. If you can wait until the first week of April I hope I can put my hands on it 💪🏻

[themattinthehatt]

No problem, let me know if I can be of any help!