FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Get max_sequence_length from model's config.json

Open chingheng113 opened this issue 2 years ago • 0 comments

In model_worker.py, line # 102-103

if hasattr(self.model.config, "max_sequence_length"):
        self.context_len = self.model.config.max_sequence_length

Should it be

if hasattr(self.model.config, "max_seq_len"):
            self.context_len = self.model.config.max_seq_len

to get the correct max sequence length from config.json?

chingheng113 avatar Jun 15 '23 09:06 chingheng113