DeepSpeedExamples icon indicating copy to clipboard operation
DeepSpeedExamples copied to clipboard

How can I find bert weights used for ZeroQuant?

Open leapingjagg-dev opened this issue 1 year ago • 0 comments

Hi I'm trying to reproduce GLUE scores reported on ZeroQuant paper. But overally almost every tasks' accuracy is lower than reported results in LKD cases. Escpecially RTE shows sharp degredation in W4/8A8 LKD case.

I assume diffrent results come out because I am using different checkpoint weights with those used for the research. The default setting is to get weights from "yoshitomo-matsubara/bert-base-uncased". How can I get weights file to reproduce at least similar numbers?

leapingjagg-dev avatar Feb 08 '23 01:02 leapingjagg-dev