Xinhao Li

Hi, thank you very much for releasing code for this inspiring work. Regarding the prediction length, say we input 10 previous frames and output 10 future frames, if our final...

Hi, thanks so much for releasing code for this inspiring work! I notice that in the ``get_sequence`` method of ``class KITTI()`` you normalize the image by ``(x / 127.5) -...

bfloat16 Training

1

Thank you for releasing code for these inspiring works! I tried to use bfloat16 for model parameters, and manually converted images and labels from float32 to bfloat16 before feeding them...

Hi, thank you so much for releasing code for this inspiring work. When I adopt distributed training on a TPU v3-128 pod for imagenet classification, with a global batch size...

Question regarding Scaling Laws on PILE

4

Thank you so much for releasing codes for this inspiring work! I found the scaling laws experiments on The PILE particularly attractive, showing the superiority of Mamba over many modern...

[Bug] adam_pax has reuse donated buffer warning

5

Hi, I noticed that when using `adam_pax` instead of `adamw` as optimizer, it will give `reuse donated buffer` warning. I am wondering if this is expected, and why the code...

TFDS Data Processing Pipline

5

Hi, I'm trying to understand some details in the TFDS data processing pipeline in your repo, and I'm confused about the following details: **In `_tfds_data_processing.py`:** (1) The `truncate_to_max_allowable_length` function truncates...

Question regarding Pathfinder and Listops performance

2

Hi, thank you for releasing code for this inspiring work! When I was trying to reproduce the results of Transformer and Linear Transformer on Pathfinder32 and Listops tasks, I encountered...

Hi, thank you so much for releasing code for this solid work! I noticed the ```config.restore_checkpoints``` flag in all config files is set to True. Does this mean this codebase...

Hi, the forward pass of our training includes passing through a nested loop with 2 layers (outer loop, inner loop). The input data X of shape [T, F] is first...