Avelina9X

Bot stops after 90 seconds - short Read, protocol is shutdown, underlying transport error

1

Afrer about 90 seconds (sometimes as little as 20, sometimes as much as 4 minutes( the bot stops responding. The console shows these errors.  I'm on WSL Ubuntu, I...

Are there any plans for supporting an explicit attention mask?

5

I've noticed that the Triton implementation supports explicit attention bias, which can be used to support arbitrary mask shapes with large negative values, however is there any planned support for...

Epsilon value of 1e-12 too small for mixed precision

1

The epsilon value of 1e-12 used in the following lines for the `first_step` and `sam_train_step` functions is too low and can cause NaN errors with training with mixed precision: `e_w...