keepsake
keepsake copied to clipboard
Display of a particular parameter/metric in `keepsake ls`
Hi!
I have a project with many parameters/metrics, so keepsake ls
is only showing a subset of them which is fine. However, if I want to compare the different experiments with regard to a parameter not displayed by keepsake, the only possibility is doing keepsake diff
with two experiments at a time (or I missed something).
Example: I want to see all experiments with a mean IoU greater than 0.1 and compare the actual values but the values are not displayed:
$ keepsake ls --filter "mean_iou > 0.1"
EXPERIMENT STARTED STATUS PARAMS BEST CHECKPOINT LATEST CHECKPOINT
951eca5 23 hours ago stopped ae_n_epochs=0 dff7f56 (step 5) 7d1bcec (step 7)
deep=false f1_score=0.3743 f1_score=0.25648
n_epochs=0
params_path=./models/train-hy...
train_limit=10000
8d2f62f 22 hours ago stopped ae_n_epochs=30 f6b8339 (step 3) a7a1b91 (step 7)
deep=true f1_score=0.25103 f1_score=0.003937
n_epochs=50
params_path=./models/train-de...
train_limit=1000
Since there is already sort & filter is there any plan to add some sort of selection parameter to keepsake ls
? If it is not too tricky I would be happy to contribute. This has already been mentioned on https://github.com/replicate/keepsake/issues/284#issuecomment-751877389
Anyway, thanks for making keepsake :confetti_ball:
Edit: I made my own query tool called sake.
Hi @Pangoraw
The Keepsake project is no longer actively maintained. If you're interested in helping maintain it, please let us know.