No dropout in model.py

[xiamengzhou]

Hi, this is great work, and thanks for releasing the code!

I found that there is no dropout in the llama models, and I wonder if it is a specific design choice? I could also have missed it, but I tried searching dropout in the code file and the paper and didn't find it.

[liaoyiqiao]

same question

[butterluo]

+1

[Lisennlp]

+1

[zaptrem]

https://github.com/h2oai/h2o-llmstudio/issues/432 appears to add dropout as an option, but it doesn't specify whether the models were pretrained with dropout disabled. Is this info public?