gemma.cpp
gemma.cpp copied to clipboard
Enhancing Utility Functions in ops.h
I have tried to use highway library in the AddFrom, MulBy, MulByConst, MulByConstAndAdd, and create_distribution functions.
Nice! Did you see any performance differences (positive or negative) @enum-class ?
Nice! Did you see any performance differences (positive or negative) @enum-class ?
@austinvhuang Du to hardware limitations, I couldn't do the performance test on the entire pipeline. I did check individual functions like MulBy, MulByConst and MulByConstAndAdd as much as fair I could. Anyway, On 12th Gen Intel Core i7-1265U with SSE, -O3, I saw roughly 2x speedup. I can share the details if you are interested:)