CogVLM
CogVLM copied to clipboard
Better set a specific version of transformers lib
System Info / 系統信息
GPU
Who can help? / 谁可以帮助到您?
No response
Information / 问题信息
- [x] The official example scripts / 官方的示例脚本
- [ ] My own modified scripts / 我自己修改的脚本和任务
Reproduction / 复现过程
Since 4.42.0, the _extract_past_from_model_output
changed the return formats
https://github.com/huggingface/transformers/blob/v4.42.0/src/transformers/generation/utils.py#L633-L650
# transformers >= 4.42.0
def _extract_past_from_model_output(self, outputs: ModelOutput, standardize_cache_format: bool = False):
past_key_values = None
cache_name = "past_key_values"
if "past_key_values" in outputs:
past_key_values = outputs.past_key_values
elif "mems" in outputs:
past_key_values = outputs.mems
elif "past_buckets_states" in outputs:
past_key_values = outputs.past_buckets_states
elif "cache_params" in outputs:
past_key_values = outputs.cache_params
cache_name = "cache_params"
# Bloom fix: standardizes the cache format when requested
if standardize_cache_format and hasattr(self, "_convert_to_standard_cache"):
batch_size = outputs.logits.shape[0]
past_key_values = self._convert_to_standard_cache(past_key_values, batch_size=batch_size)
return cache_name, past_key_values
https://github.com/huggingface/transformers/blob/v4.41.2/src/transformers/generation/utils.py#L613-L626
# transformers < 4.42.0
def _extract_past_from_model_output(self, outputs: ModelOutput, standardize_cache_format: bool = False):
past_key_values = None
if "past_key_values" in outputs:
past_key_values = outputs.past_key_values
elif "mems" in outputs:
past_key_values = outputs.mems
elif "past_buckets_states" in outputs:
past_key_values = outputs.past_buckets_states
# Bloom fix: standardizes the cache format when requested
if standardize_cache_format and hasattr(self, "_convert_to_standard_cache"):
batch_size = outputs.logits.shape[0]
past_key_values = self._convert_to_standard_cache(past_key_values, batch_size=batch_size)
return past_key_values
Better set the specific version of transformers in requirements.txt
and highlight in installation readme, or change the modeling code here https://huggingface.co/THUDM/cogagent-chat-hf/blob/d519da3b191401234f4bd86ce1c287c61bc276a3/modeling_cogagent.py#L878-L880:
model_kwargs["past_key_values"] = self._extract_past_from_model_output(
outputs, standardize_cache_format=standardize_cache_format
)
to
_, model_kwargs["past_key_values"] = self._extract_past_from_model_output(
outputs, standardize_cache_format=standardize_cache_format
)
with the latest version of transformers lib.
Expected behavior / 期待表现
Fix the version bug