minGPT icon indicating copy to clipboard operation
minGPT copied to clipboard

Add dtype support

Open younesbelkada opened this issue 1 year ago • 0 comments

Motivation

People may want to use minGPT in different precisions (fp16, fp32, bf16). This PR integrates this feature to add this possibility to the library. Only floating point precisions are supported here. Users can:

  • Create a model with a different precision than fp32

Added features

  • memory footprint tracking of minGPT models
  • Added a new test for checking the memory footprint of casted models

Other

Not sure if a pretrained model can be loaded in the desired dtype (needs to add kwargs on from_pretrained method probably)

@karpathy

younesbelkada avatar Aug 05 '22 07:08 younesbelkada