[text_classification.ipynb] Model initialized twice during hyperparameter search

[don-tpanic]

trafficstars

Thank you for the helpful examples!

I have a question regarding: https://github.com/huggingface/notebooks/blob/main/examples/text_classification.ipynb

Specifically, under the Hyperparameter search section, I wonder if it is expected behavior that the model is initialized twice.

First time -

trainer = Trainer(
    model_init=model_init,
    args=args,
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset[validation_key],
    tokenizer=tokenizer,
    compute_metrics=compute_metrics
)

Second time -

best_run = trainer.hyperparameter_search(n_trials=10, direction="maximize")

I would expect the model would only be initialized once when the search begins. Currently, RAM usage would double which is not desirable if the model is huge.

Any pointers?

Thanks!