artyom-beilis
Many warnings with PyTorch 2.2.1: aten::_foreach_lerp_.Scalar, aten::_foreach_mul_.Scalar, aten::_foreach_sqrt and others
/home/inetstar/Kamenev/programming/lib/python3.11/site-packages/torch/optim/adam.py:520: UserWarning: The operator 'aten::foreach_lerp.Scalar' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at /home/inetstar/Kamenev/programming/pytorch_dlprim/src/tensor_ops.cpp:311.) torch.foreach_lerp(device_exp_avgs, device_grads, 1 - beta1) /home/inetstar/Kamenev/programming/lib/python3.11/site-packages/torch/optim/adam.py:522: UserWarning: The operator 'aten::foreach_mul.Scalar' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at /home/inetstar/Kamenev/programming/pytorch_dlprim/src/tensor_ops.cpp:311.) torch.foreach_mul(device_exp_avg_sqs, beta2) /home/inetstar/Kamenev/programming/lib/python3.11/site-packages/torch/optim/adam.py:523: UserWarning: The operator 'aten::foreach_addcmul.Scalar' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at /home/inetstar/Kamenev/programming/pytorch_dlprim/src/tensor_ops.cpp:311.) torch.foreach_addcmul(device_exp_avg_sqs, device_grads, device_grads, 1 - beta2) /home/inetstar/Kamenev/programming/lib/python3.11/site-packages/torch/optim/adam.py:579: UserWarning: The operator 'aten::_foreach_sqrt' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at /home/inetstar/Kamenev/programming/pytorch_dlprim/src/tensor_ops.cpp:311.) exp_avg_sq_sqrt = torch._foreach_sqrt(device_exp_avg_sqs) /home/inetstar/Kamenev/programming/lib/python3.11/site-packages/torch/optim/adam.py:581: UserWarning: The operator 'aten::foreach_div.ScalarList' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at /home/inetstar/Kamenev/programming/pytorch_dlprim/src/tensor_ops.cpp:311.) torch.foreach_div(exp_avg_sq_sqrt, bias_correction2_sqrt) /home/inetstar/Kamenev/programming/lib/python3.11/site-packages/torch/optim/adam.py:582: UserWarning: The operator 'aten::foreach_add.Scalar' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at /home/inetstar/Kamenev/programming/pytorch_dlprim/src/tensor_ops.cpp:311.) torch.foreach_add(exp_avg_sq_sqrt, eps) /home/inetstar/Kamenev/programming/lib/python3.11/site-packages/torch/optim/adam.py:583: UserWarning: The operator 'aten::foreach_addcdiv.ScalarList' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at /home/inetstar/Kamenev/programming/pytorch_dlprim/src/tensor_ops.cpp:311.) torch.foreach_addcdiv(device_params, device_exp_avgs, exp_avg_sq_sqrt, step_size)
Yes, this something happened starting from 2.2 (IIRC)
I try to understand how to handling because it is crazy to re-implement dozen of operators that can be trivially done using other operators.
