Charlie Lin

Dangling quantizelinear from horizontal fusion, BERT and DistilGPT2

2

* Found during Inference Model Review meeting * Seen in bert_base_cased and distilgpt2_fp16 run with our `--fp8` flag and probably also `--int8` ``` @24 = gpu::code_object[code_object=8920,symbol_name=mlir_quantizelinear_quant_dot_dequantizelinear_add_add,global=1769472,local=256,](@18,@21,@23,@15,@22) -> half_type, {64, 384,...

* Found during Inference Model Review meeting * Seen in bert_base_cased and distilgpt2_fp16 run with our `--fp8` flag and probably also `--int8` ``` @12 = hip::hip_copy_literal[id=main:@literal:17] -> half_type, {768, 2304},...

MXFP4: rocMLIR compilation remove unpack_fp4 and bugfixes

3

## Motivation * Better handing of MXFP4 for rocMLIR compilation ## Technical Details * Depends on #4343 * `pack_fp4` and `unpack_fp4` changed to always pack/unpack on the fastest dimension. *...

MXFP4 - rocMLIR integration

4

## Motivation * Part of https://github.com/ROCm/AMDMIGraphX-internal/issues/149 ## Technical Details * Requires https://github.com/ROCm/rocMLIR/tree/packFp4 from rocMLIR to work on MI350. * This will pass CI since CI doesn't run on a MI350...

`generic_float` for Float8E8M0

1

## Motivation * Introduce Float8E8M0 type within MIGraphX for better MXFP4 optimizations and to use hipblaslt mxfp4 kernels. ## Technical Details ## Changelog Category - - [ ] Added: New...

### DOR (Definition of Ready) Ready. ### Description * Need to support `E8M0` type for better handling of MX block scales in MIGX. * Current `generic_float` implementation doesn't allow for...

### DOR (Definition of Ready) When rocm 7.0 is released and MIGraphX CI build is updated for it. ### Description Update the pack_fp4 and unpack_fp4 GPU kernels to use the...

* Currently we only handle int8 mixed precision, uint8 activations and int8 weights, in GEMMS in the parser. * There needs to be an update that adds a pass or...

* Currently the `allocate_gpu`, `to_gpu`, `from_gpu` and `gpu_sync` functions are exposed through the python API. * We should make it such that these functions are hidden behind the `target.copy_to()` and...

split multiple of the same dynamic dimension for dynamic batch

1

An example case would be when we have a square image. Such that the height and width are the same but dynamic.