Max Balandat
Max Balandat
So 10K evaluations are a lot - that’s above the typical range that Ax was designed for and you’ll likely run into scalability issues if you’re using our standard models....
I can confirm that `jit_compile=False` fixes things - I'll look into the what the speedup from this is.
I guess I would still like to see this covered by a unit test though :)
This is a great suggestion, and IIRC @j-wilson has played around with this a bit before. Not sure what state that is in and whether it makes sense for him...
> I'm hopeful that we'll be able to use e.g. torch.jit.script to compile these closures and expedite training. Sam has been having some good success using torchdynamo/torchinductor, would be interesting...
This can happen if the training data results in ill-conditioned covariances. Do you have a lot of repeated (or almost repeated) points? Did you normalize / standardize the training inputs...
> I am not using torch.double datatype. Using `torch.double` may take care of the issues, I would try that first. In general it's very hard to get GPs to work...
> Perhaps because we are using a CPU to run our optimization so float dtypes are equivalent to double dtypes as you've said? Oh no that is not the case,...
> It's true this isn't really "true Bayesian optimization", but I think there is a community that will find this useful. It kinda feels like botorch should have this. Agreed,...