Joonhyung Lee/이준형

Create `set_default_dtype` function to allow setting float32 as the default data type.

2

### Description I would like to request a function that changes the default data type of CuPy arrays. Currently, CuPy follows Numpy's convention in always using float64 as the default...

Hello. I was going through the attention implementation and had a question about the operation by the einsum function. I am not very familiar with the Einstein summation convention and...

Hello. I would like to ask for new CUDA versions. According to the documentation, adding a new minor version of CUDA is a relatively simple matter. As CUDA 11.4 has...

Add dynamic channel expansion.

11

Hello. I have been looking at the current implementation and I have found that it uses .repeat() on CPU. However, I would like to propose using .expand() inside the ssim()...

@ChihebTrabelsi In the current implementation of ComplexInit in init.py, the scale factor uses the following. `if self.criterion == 'glorot': s = 1. / (fan_in + fan_out) elif self.criterion == 'he':...

[Feature Request] Breaking from Python syntax (slightly)

2

## Request Hello. Thank you for the awesome project. As a new project, I would suggest that Mojo breaks with Python on some syntactical points where Python only maintains undesirable...

[Feature Request] Compressed-tile Matrix Multiply for autoregressive LLM inference

1

### Is your feature request related to a problem? Please describe. I would like to request the implementation of a compressed tiled matrix multiply operator for use in large language...

[Question] Is pre-training with FP32 possible?

2

Hello, I am currently trying out pre-training and I was curious if the data type used for pre-training could be configured from using `torch.bfloat16` to `torch.float32`. I would like to...

[FEA] Add support for CUDA compressed memory.

1

**Is your feature request related to a problem? Please describe.** I would like to ask for RMM support of CUDA compressed memory, a feature available in the A100 and H100...

[20230723] Weekly AI ArXiv 만담 시즌2 - 23회차

4

하정우님께서 코로나에 걸려 :worried::worried::worried: 오늘 발표는 제가 리드를 맡게 되었습니다. Weekly ArXiv Talk은 매주 가장 최신 뉴스 및 연구 동향에 대해 가볍게 토의하는 자리로 공유한 모든 내용이 정확하지 않을 수...