scvi-tools icon indicating copy to clipboard operation
scvi-tools copied to clipboard

Add changes for tracking scib-metrics in autotune

Open martinkim0 opened this issue 6 months ago • 0 comments

Here's an example code snippet for how this works with SCVI:

import scvi
from scvi.autotune import ModelTuner
from scvi.train._scib_callback import ScibCallback

adata = scvi.data.synthetic_iid()
model_cls = scvi.model.SCVI
model_cls.setup_anndata(adata, batch_key="batch", labels_key="labels")

tuner = ModelTuner(model_cls)
tuner.fit(
    adata,
    use_defaults=True,
    metric="validation Batch correction",  # necessary
    train_kwargs={"callbacks": [ScibCallback(stage="validation")]},  # necessary
    validate_metrics=False,  # necessary
)

Code is not nice yet as the TrainingPlan has to accumulate and store the raw counts, latent representation, batches, and labels for this to work.

martinkim0 avatar Feb 16 '24 19:02 martinkim0