gin-config icon indicating copy to clipboard operation
gin-config copied to clipboard

Question about using multiple gin files for multiple models with clashing parameters

Open snnithya opened this issue 1 month ago • 0 comments

Hi, I have trained two models (say A and B) separately, each with their respective gin files. The problem now is that I want to load both models within the same script for inference since the output of model A goes as input into model B. However, the gin files of A and B have certain common parameters defined with different values. For instance, the function to load the dataset in the Dataset class has different preprocessing functions for each model and are both referred to with the parameter Dataset.task_fn. Thus if I use gin.configure_files_and_bindings to load both files, the Dataset.task_fn value gets overwritten by one of the gin files. Is there some way around this?

One solution I could think of is defining the common parameters within different gin files for instance in model A's gin file, defining the parameter as modelA/Dataset.task_fn and similarly modelB/Dataset.task_fn for modelB and thus in python, I would load the models within their respective scopes. This seems kind of hacky and would involve me changing the gin files. I was wondering if there is a better solution I can use?

Thanks in advance!!

snnithya avatar May 10 '24 16:05 snnithya