zhujiem

Question signed sqrt normalization

1

Is it correct to use torch.sqrt(outputs + 1e-5) since outputs may have negative values? Thanks! https://github.com/dasguptar/bcnn.pytorch/blob/master/bcnn/model.py#L32 `outputs = torch.sign(outputs) * torch.sqrt(outputs + 1e-5) # signed square root normalization`

+ Wukong: [Towards a Scaling Law for Large-Scale Recommendation](https://arxiv.org/html/2403.02545v1) + [TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest](https://arxiv.org/pdf/2306.00248) + DPN: [Deep Pattern Network for Click-Through Rate Prediction](https://arxiv.org/pdf/2404.11456v1) +...

Huggingface Datasets: ``` dataset = load_dataset("parquet", data_files={split: data_blocks}, split=split) super().__init__(dataset=dataset, num_workers=8, batch_size=self.batch_size) ```

Monarch & PixelFly based MLP layer efficiency testing

3

Here I post some efficiency testing numbers for Monarch based MLP v.s. vanilla nn.Linear based MLP. I found that Monarch is best suitable for MLPs in Transformer architectures, which generally...

Invitation to contribute to LOGPAI

2

-Invitation to contribute to LOGPAI- Hi, this is an invitation from LOGPAI, which is an open-source project towards building log anlaytics solutions powered by AI. Would you like to contribute...

Hi, Thanks for the open source code of TransAct! Recently, I have integreted TransAct to [our FuxiCTR library](https://github.com/reczoo/FuxiCTR/tree/main/model_zoo/TransAct). But I found some confusing results when applying grid search on the...