Trangle Heshvp

 The main work is on sample construction, which has changed from the original estimation of (x, yl)+(x, yw) to (x+yl, yl)+(x+yl, yw)+(x+yw, yl)+(x+yw, yw), resulting in a significant increase...

> > This has been discussed in [multiple](https://github.com/huggingface/trl/pull/1265) [github](https://github.com/h2oai/h2o-llmstudio/issues/580) [issues](https://github.com/huggingface/trl/issues/1294), and I believe the answer stems from the discussion that Huggingface had with the IPO authors [here](https://huggingface.co/blog/pref-tuning) "After consulting with...