gemma.cpp icon indicating copy to clipboard operation
gemma.cpp copied to clipboard

Enhancing Utility Functions in ops.h

Open enum-class opened this issue 1 year ago • 2 comments

I have tried to use highway library in the AddFrom, MulBy, MulByConst, MulByConstAndAdd, and create_distribution functions.

enum-class avatar Mar 19 '24 00:03 enum-class

Nice! Did you see any performance differences (positive or negative) @enum-class ?

austinvhuang avatar Mar 20 '24 01:03 austinvhuang

Nice! Did you see any performance differences (positive or negative) @enum-class ?

@austinvhuang Du to hardware limitations, I couldn't do the performance test on the entire pipeline. I did check individual functions like MulBy, MulByConst and MulByConstAndAdd as much as fair I could. Anyway, On 12th Gen Intel Core i7-1265U with SSE, -O3, I saw roughly 2x speedup. I can share the details if you are interested:)

enum-class avatar Mar 21 '24 01:03 enum-class