Masaki Kozuki comments

Results 167 comments of


                                            Masaki Kozuki

Optuna Suggests the Same Parameter Values in a lot of Trials (Duplicate Trials that Waste Time and Budget)

By its nature, TPESampler tends to sample similar values with the number of trials increasing since it narrows the search space based on Bayesian optimization with the number of trials...

Export version

we're thinking of switching to Calendar Versioning in a month: https://github.com/NVIDIA/apex/pull/1648

[mta] Backward of unary foreach functions

regarding inplace ops, I locally confirmed the version seems to be bumped appropriately ``` In [8]: tensors = [torch.randn(3).requires_grad_() for _ in range(2)] In [9]: print([t._version for t in tensors])...

[mta] Backward of unary foreach functions

Added a codegen and a version bump

[mta] Backward of unary foreach functions

@pytorchbot merge (knowing this won’t work this time but want to trigger more jobs)

[mta] Backward of unary foreach functions

@pytorchbot merge

[mta] implement `_foreach_pow`

@pytorchbot rebase

[mta] implement `_foreach_pow`

@pytorchbot label ciflow/trunk

[mta] implement `_foreach_pow`

@pytorchbot merge

Small learning rate with `capturable=True` causes Adam optimizer to blow up model paraemters.

@qq387426 could you provide a minimal reproducible snippet? I ran the snippet below with torch==2.1.0a0+gitb132220 on 3070Ti but didn't see inf/nan/large values as in the bottom ```python from itertools import...