Kimiko

Adding FasterTransformer support

2

https://github.com/NVIDIA/FasterTransformer Implementing FasterTransformer may increase speed, reduce memory usage for many models

@merrymercy Check this github issue

Add Prodigy, SophiaG optimizers

3

# Description https://arxiv.org/pdf/2306.06101.pdf https://arxiv.org/abs/2305.14342 ## Motivation and Context ## How has this been tested? No ## Screenshots (if appropriate) ## Types of changes ## Social Handles (Optional)