sls

sls copied to clipboard

Independence of the step-size and stochastic gradient ?

2

Dear authors, Thanks for this work According to the paper, Appendix F.1 in page 25: "To enforce independence of the step-size and stochastic gradient, we perform a backtracking line-search at...

Algue-Rythme

First, thanks for the nice work. I am excited to try out this optimizer. I noticed that in `get_optimizer` you have the following line https://github.com/IssamLaradji/sls/blob/318e9b569bdda96a925685030634238f043e3964/src/optimizers/__init__.py#L27 Where does this heuristic come...

chanshing

Hi. I have this NN model : ``` N, D_in, H, D_out = X.shape[0], X.shape[1], inh , Y.shape[1] ; model = nn.Sequential(OrderedDict([ ('fc1', nn.Linear(D_in, H)), ('Tanh', nn.Tanh()), ('fc2', nn.Linear(H, D_out))]))...

automataIA

SLS and parameter groups for larger datasets?

13

I'm hitting an issue though in using/testing as the code seems to assume no parameter groups? (from utils.py) def get_grad_list(params): return [p.grad for p in params] this fails b/c p.grad...

lessw2020

Tensorflow version

1

Hi, this is not the actual issue in this repository. I just want to ask that do you plane to provide the Tensorflow version in the future? Since I'm developing...

SunnerLi

Documentation `n_batches_in_epoch` incorrect argument name

1

Just letting you know in the README.md you reference an argument with the name `n_batches_in_epoch` but within the code this argument is actually called `n_batches_per_epoch`.

liamcripwell