Yongxing Dai

icon indicating a website link https://sikastar.github.io/ icon indicating an email [email protected]

icon company Peking University icon location Beijing China icon location I am now a fifth-year PhD student at National Engineering Lab for Video Technology in Peking University, Beijing, China

Results 2 comments of Yongxing Dai

pairwise loss in pytorch code

> @bfan @caozhangjie > I add the weight in pytorch version(without c). > > ``` > def pairwise_loss(outputs1,outputs2,label1,label2): > similarity = Variable(torch.mm(label1.data.float(), label2.data.float().t()) > 0).float() > dot_product = torch.mm(outputs1, outputs2.t())...

SFT FlashAttention questions

> from verl/workers/fsdp_workers.py. > > ``` > torch_dtype = fsdp_config.get("model_dtype", None) > if torch_dtype is None: > torch_dtype = torch.float32 if self._is_actor else torch.bfloat16 > else: > torch_dtype = PrecisionType.to_dtype(torch_dtype)...