Constantin Dumitrascu comments

Results 50 comments of


                                            Constantin Dumitrascu

Investigate triton

Marking the items prior to Feb 29th as "closed".

Benchmarking different RoPE impls

I apologize for our delay in response. In order to help surface current, unresolved issues, we are closing tickets prior to February 29. Please reopen your ticket if you are...

FSDP Overlap Investigation

Marking the items prior to Feb 29th as "closed".

Debugging the Spikyness of the 7B

Marking the items prior to Feb 29th as "closed".

Ensuring Data Order Tracking for Reproducibility

Marking the items prior to Feb 29th as "closed".

[Inquire of MMLU Performance]

It appears that the question has been answered. Please reopen if still relevant.

Better Checkpoint Management

Marking the items prior to Feb 29th as "closed".

Try adding intermediate layer losses

Marking the items prior to Feb 29th as "closed".

Ablate Kaiming init vs Mitch init vs our own init at 7B

Marking the items prior to Feb 29th as "closed".

Try longer warmup (5k) at the 7B scale with mitch init, normal init and fan-in init

Marking the items prior to Feb 29th as "closed".