zhengjia comments

Results 16 comments of


                                            zhengjia

Difference between hip_bfloat16 and __hip_bfloat16?

> Hi @pcmoritz, you can now directly convert between the two data types in the latest ROCm release. btw, can you provide the public image link for latest rocm build...

Difference between hip_bfloat16 and __hip_bfloat16?

another thing, if need `__float2bfloat16()`, `__hip_bfloat16` has built-in apis, while `hip_bfloat16` doesn't

split ReplicatedLinear used in MLA prefill computing along hidden_states[0] to save duplicated computing on all devices

serving bench results update: ## baseline without use_dp_linear ```yml # prefill ============ Serving Benchmark Result ============ Backend: sglang Traffic request rate: inf Max reqeuest concurrency: 32 Successful requests: 200 Benchmark...

split ReplicatedLinear used in MLA prefill computing along hidden_states[0] to save duplicated computing on all devices

> Hi, please let me know when it's ready for review. Thanks! hi, @zhyncs many thanks for review. bench test covered just few isl/osl/num_promts on h20/mi30x

[HIPIFY] How to hipify the programs based nvidia cutlass

any update on cutlass supporting recently ?

Codebase is messy.

better to read https://github.com/FasterDecoding/Medusa/tree/main this code arch first, much clear