FastChat
FastChat copied to clipboard
Get max_sequence_length from model's config.json
In model_worker.py, line # 102-103
if hasattr(self.model.config, "max_sequence_length"):
self.context_len = self.model.config.max_sequence_length
Should it be
if hasattr(self.model.config, "max_seq_len"):
self.context_len = self.model.config.max_seq_len
to get the correct max sequence length from config.json?