keepsake icon indicating copy to clipboard operation
keepsake copied to clipboard

Display of a particular parameter/metric in `keepsake ls`

Open Pangoraw opened this issue 3 years ago • 1 comments

Hi!

I have a project with many parameters/metrics, so keepsake ls is only showing a subset of them which is fine. However, if I want to compare the different experiments with regard to a parameter not displayed by keepsake, the only possibility is doing keepsake diff with two experiments at a time (or I missed something).

Example: I want to see all experiments with a mean IoU greater than 0.1 and compare the actual values but the values are not displayed:

 $ keepsake ls --filter "mean_iou > 0.1"
EXPERIMENT  STARTED       STATUS   PARAMS                            BEST CHECKPOINT   LATEST CHECKPOINT
951eca5     23 hours ago  stopped  ae_n_epochs=0                     dff7f56 (step 5)  7d1bcec (step 7)
                                   deep=false                        f1_score=0.3743   f1_score=0.25648
                                   n_epochs=0
                                   params_path=./models/train-hy...
                                   train_limit=10000

8d2f62f     22 hours ago  stopped  ae_n_epochs=30                    f6b8339 (step 3)  a7a1b91 (step 7)
                                   deep=true                         f1_score=0.25103  f1_score=0.003937
                                   n_epochs=50
                                   params_path=./models/train-de...
                                   train_limit=1000

Since there is already sort & filter is there any plan to add some sort of selection parameter to keepsake ls ? If it is not too tricky I would be happy to contribute. This has already been mentioned on https://github.com/replicate/keepsake/issues/284#issuecomment-751877389

Anyway, thanks for making keepsake :confetti_ball:

Edit: I made my own query tool called sake.

Pangoraw avatar Apr 09 '21 07:04 Pangoraw

Hi @Pangoraw

The Keepsake project is no longer actively maintained. If you're interested in helping maintain it, please let us know.

zeke avatar Mar 15 '22 20:03 zeke