Jeffrey Tsz-hang Wong comments

Repositories
Issues
Comments

Results 2 comments of


                                            Jeffrey Tsz-hang Wong

Can not reproduce the results of Llama-3-8B

No, I didn't manage to reproduce their results

1 / Log(loss)

I believe they used XX^T without dividing the number of samples. The truncation loss is quite big in that case and should avoid getting the negative problem. However, I didn't...