Expertium

Alright, I estimated alphas and betas for all 10k users using the code I showed here: https://github.com/open-spaced-repetition/srs-benchmark/issues/157#issuecomment-2600305958 Median alpha = 43.1 Median beta = 3.2 Mean alpha = 53.4 Mean...

Unrelated, but @1DWalker, can you show me your training loop for the LSTM?

Thank you, I'll tkae a look. Btw, for hyperparameter tuning I recommend this: https://ax.dev/docs/bayesopt.html Here's an example from training a Transformer (I simplified it a little bit): ``` from ax.service.ax_client...

Later I'll try [this idea](https://github.com/open-spaced-repetition/srs-benchmark/issues/157#issuecomment-2600119248) but with the LSTM. I won't be able to estimate RMSE, but I should be able to estimate log loss and AUC.

@1DWalker nevermind, I'm using the latest benchmarking code and I get an error: ``` File "C:\Users\Andrew\srs-benchmark2\other.py", line 2251, in iter sequences, delta_ts, labels, seq_lens, weights = batch ValueError: not enough...

`python other.py --model LSTM --processes 4`

Yeah, I think I forgot to specify the correct path to the optimizer (the code dosn't work with the default path for me) Anyway, if you plan to make a...

@1DWalker I wonder if you could do something like the Chinchilla Scaling Laws but for this dataset and with only the number of parameters of the LSTM (or whatever model...

I forgot @user1823 IMO this is a serious problem. "If we don't solve it, we're f###ed" kind of serious. I'd like to have as many people pay attention to it...

My idea that I shared on Discord: How about this: 1) Calculate retention within a given batch, let's call it `retention_batch`. 2) Use an exponential moving average, like this: `retention_ma...