text-generation-inference

text-generation-inference copied to clipboard

Feature request

TGI should read the config.json and apply the rope scaling and factor from the config.json parameter.

Motivation

Many inference engines auto-apply the rope scaling and rope scaling factor based on the parameter in config.json.

Your contribution

Providing how multiple inference engine applies the scaling based on the config.json