Added A10 to mfu.h

Open tiehexue opened this issue 1 year ago • 1 comments

values in mfu.h added for A10 are copied from https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a10/pdf/a10-datasheet.pdf

With two A10 or only one A10, it shows around "43.4% bf16 MFU".

BTW: it tooks 24 hours for two A10 to finish the training on fineweb10B.

Jun 20 '24 08:06 tiehexue

Is Ampere a different class from Consumer and Datacenter? cc @ngc92 who originally wrote this code and did some of the researching.

Jun 20 '24 17:06 karpathy