Richard Zou

Results 218 comments of Richard Zou

Exception handling in linux binaries seem off

There's also a related problem (which was discussed offline): the PyTorch cu102 binaries don't include the `_ZNSt19basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev` symbol, but the PyTorch cpu/cu113/cu116 binaries do. On some systems, libstdc++.so.6 doesn't actually...

Excise dependency on networkx

Optional should be fine for now

Add an Ensemble Module that is constructed from a list of Modules and encapsulates the necessary state

This seems convenient to have. I am not sure if this would go into functorch or in torch.nn in the long-term state, but we can certainly toss something like this...

Add an Ensemble Module that is constructed from a list of Modules and encapsulates the necessary state

> Also this seems to be very vmap specific? Are you suggesting that we should put the nn.Ensemble API into functorch because it is vmap specific?

No Batching rules for aten::_linalg_solve_ex, aten::linalg_solve, aten::linalg_solve_ex, and aten::_linalg_slogdet causes significant slowdown for per-sample gradients with torch.linalg.slogdet

cc @samdow -- were these the ones you were planning on adding? I haven't checked if these fall into the low-hanging-fruit category

No Batching rules for aten::_linalg_solve_ex, aten::linalg_solve, aten::linalg_solve_ex, and aten::_linalg_slogdet causes significant slowdown for per-sample gradients with torch.linalg.slogdet

@samdow did https://github.com/pytorch/pytorch/pull/82177 cover all of these?

No Batching rules for aten::_linalg_solve_ex, aten::linalg_solve, aten::linalg_solve_ex, and aten::_linalg_slogdet causes significant slowdown for per-sample gradients with torch.linalg.slogdet

I added linalg_lu_solve yesterday, could you reinstall the latest pytorch nightly and try again?

No Batching rules for aten::_linalg_solve_ex, aten::linalg_solve, aten::linalg_solve_ex, and aten::_linalg_slogdet causes significant slowdown for per-sample gradients with torch.linalg.slogdet

I think I know what is going on here, will fix soon. EDIT: fix over at https://github.com/pytorch/pytorch/pull/86815 . Thanks as always for reporting bugs, @AlphaBetaGamma96. It doesn't look like we've...

Performant way to initialize an ensemble of models

In order to combine the models together for ensembling with vmap, each model must call exactly the same sequence of PyTorch operations (otherwise, vmap will not work). So in that...

How to update the original model parameters after calling make_functional?

Couldn't you do ``` def optstep(model, jacobians): with torch.no_grad(): for i, param in enumerate(model.parameters()): param.add_(jacobians[i], alpha=-learning_rate) ``` ? (Also, you might want to try functorch.jacrev instead of torch.autograd.functional.jacobian -- it...