bigcodebench icon indicating copy to clipboard operation
bigcodebench copied to clipboard

[REQUEST] Remove Reflection-Llama-3.1-70B

Open wakamex opened this issue 1 year ago • 0 comments
trafficstars

Rationale

It's been proven to be a fraud. Their CEO has repeatedly lied and not fully come clean about:

  1. Uploading weights identical to Llama 3
  2. Faking API responses using Sonnet and ChatGPT

Sources: https://x.com/MaziyarPanahi/status/1838559480658710982 https://x.com/Yuchenj_UW/status/1833627813552992722

The repo referenced in the request to add Reflection has since been deleted.

The repo that hosts the re-uploaded weights warns the original benchmarks can't be trusted: image

The founder now admits the model doesn't work: https://x.com/mattshumer_/status/1842313328166907995

Proof the initial released weights are just LoRa applied to Llama 3: https://www.reddit.com/r/LocalLLaMA/comments/1fb6jdy/reflectionllama3170b_is_actually_llama3/

wakamex avatar Oct 07 '24 19:10 wakamex