InternLM-XComposer
InternLM-XComposer copied to clipboard
InternLM
NotImplementedError when running official example code of InternLM-XComposer-2.5-Reward
When I run the official example code, I noticed there are warnings:
/home/tiger/miniconda3/envs/internlm/lib/python3.11/site-packages/torch/nn/modules/module.py:2397: UserWarning: for vision_model.encoder.layers.10.mlp.fc2.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
warnings.warn(
/home/tiger/miniconda3/envs/internlm/lib/python3.11/site-packages/torch/nn/modules/module.py:2397: UserWarning: for vision_model.encoder.layers.10.mlp.fc2.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
warnings.warn(
/home/tiger/miniconda3/envs/internlm/lib/python3.11/site-packages/torch/nn/modules/module.py:2397: UserWarning: for vision_model.encoder.layers.10.layer_norm2.weight: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
warnings.warn(
/home/tiger/miniconda3/envs/internlm/lib/python3.11/site-packages/torch/nn/modules/module.py:2397: UserWarning: for vision_model.encoder.layers.10.layer_norm2.bias: copying from a non-meta parameter in the checkpoint to a meta parameter in the current model, which is a no-op. (Did you mean to pass `assign=True` to assign items in the state dictionary to their corresponding key in the module instead of copying them in place?)
which may lead to further errors:
Traceback (most recent call last):
File "code/videogen/eval/InternLM-XComposer-2.5-Reward/inference.py", line 4, in <module>
model = AutoModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/tiger/.local/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 559, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/tiger/.local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 261, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/tiger/.local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4371, in from_pretrained
dispatch_model(model, **device_map_kwargs)
File "/usr/local/lib/python3.11/dist-packages/accelerate/big_modeling.py", line 498, in dispatch_model
model.to(device)
File "/home/tiger/.local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3142, in to
return super().to(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1340, in to
return self._apply(convert)
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 900, in _apply
module._apply(fn)
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 900, in _apply
module._apply(fn)
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 900, in _apply
module._apply(fn)
[Previous line repeated 3 more times]
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 927, in _apply
param_applied = fn(param)
^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1333, in convert
raise NotImplementedError(
NotImplementedError: Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device.
This error is raised when you use 'device_map='auto' to load the model. You may try to load the model like this:
model = AutoModelForCausalLM.from_pretrained(ckpt_path, torch_dtype=torch.float16, trust_remote_code=True).cuda().eval()
Hi @yuhangzang
Thanks for your quickly reply, this error is happed when I use device_map='cuda'
My environment is built follow the official instruction:
- `Accelerate` version: 1.2.1
- Platform: Linux-5.4.143.bsk.7-amd64-x86_64-with-glibc2.36
- `accelerate` bash location: /home/tiger/miniconda3/envs/intern/bin/accelerate
- Python version: 3.11.2
- Numpy version: 1.26.4
- PyTorch version (GPU?): 2.5.1 (True)
- PyTorch XPU available: False
- PyTorch NPU available: False
- PyTorch MLU available: False
- PyTorch MUSA available: False
- System RAM: 2015.23 GB
- GPU type: NVIDIA A100-SXM4-80GB
with pip install transformers==4.33.2 timm==0.4.12 sentencepiece==0.1.99 gradio==4.13.0 markdown2==2.4.10 xlsxwriter==3.1.2 einops
and I use the official example code on Huggingface:
import torch
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained(
"internlm/internlm-xcomposer2d5-7b-reward",
device_map="cuda",
torch_dtype=torch.float16,
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-xcomposer2d5-7b-reward", trust_remote_code=True)
model.tokenizer = tokenizer
chat_1 = [
{"role": "user", "content": 'I want to buy a car from the input image, analyze the advantages and weaknesses.'},
{"role": "assistant", "content": "The car in the image is a Mercedes-Benz G-Class. This is a luxury SUV known for its ruggedness, off-road capability, and iconic boxy design. Here's an analysis of its advantages and weaknesses: Advantages: Off-road capability: The G-Class is built on a robust ladder frame chassis and has a powerful engine, making it highly capable in off-road conditions. It features a low-range transfer case, locking differentials, and a high ground clearance. Luxury and comfort: Despite its rugged exterior, the G-Class offers a luxurious interior with high-quality materials, comfortable seats, and advanced technology features. Iconic design: The distinctive boxy shape of the G-Class has made it a cult classic. It's instantly recognizable and has a timeless appeal. Strong resale value: Due to its exclusivity and desirability, the G-Class tends to hold its value well over time. Weaknesses: Fuel efficiency: The G-Class is not known for its fuel efficiency, especially with the larger engine options. Ride quality: The firm suspension is designed for off-road performance, which can make the ride a bit bumpy on paved roads. Price: The G-Class is a very expensive vehicle, even for a luxury SUV. Size and maneuverability: The large size of the G-Class can make it challenging to park and maneuver in tight spaces."}
]
chat_2 = [
{"role": "user", "content": 'I want to buy a car from the input image, analyze the advantages and weaknesses.'},
{"role": "assistant", "content": "Based on the image, it appears to be a Ferrari F8 Tributo. Advantages: Performance: Ferraris are renowned for their exceptional performance. The F8 Tributo is no exception, boasting a powerful twin-turbocharged V8 engine that delivers exhilarating acceleration and top speed. Handling: Ferraris are known for their precise handling and agility. The F8 Tributo continues this tradition, with advanced aerodynamics and sophisticated suspension systems ensuring a thrilling driving experience. Design: Ferrari is synonymous with iconic and head-turning designs. The F8 Tributo is a testament to this, featuring sleek lines, aggressive styling, and a captivating presence. Exclusivity: Owning a Ferrari is a status symbol. It represents luxury, exclusivity, and a passion for high-performance automobiles. Weaknesses: Price: Ferraris come with a hefty price tag. The F8 Tributo is no exception, making it an investment for those with significant financial resources. Fuel Efficiency: High-performance sports cars like the F8 Tributo are not known for their fuel efficiency. You can expect lower miles per gallon compared to everyday vehicles. Maintenance Costs: Owning a Ferrari comes with associated maintenance costs, which can be higher than those for regular cars. Practicality: The F8 Tributo is primarily a two-seater sports car, making it less practical for everyday use or carrying passengers. Ride Comfort: While the F8 Tributo offers a thrilling driving experience, its stiff suspension might not be ideal for long-distance comfort."}
]
image = ['eval/InternLM-XComposer-2.5-Reward/cars1.jpg']
hd_num = 9
# get reward score for a single chat
with torch.autocast(device_type='cuda', dtype=torch.float16):
score1 = model.get_score(chat_1, image, hd_num=hd_num)
score2 = model.get_score(chat_2, image, hd_num=hd_num)
print("score1: ", score1)
print("score2: ", score2)
# >>> score1: 5.76
# >>> score2: -2.84375
# batch inference, get multiple scores at once
with torch.autocast(device_type='cuda', dtype=torch.float16):
scores = model.get_scores([chat_1, chat_2], [image, image], hd_num=hd_num)
print("scores: ", scores)
# >>> scores: [5.76171875, -2.845703125]
# compare whether chat_1 is better than chat_2
with torch.autocast(device_type='cuda', dtype=torch.float16):
compare_res = model.compare(chat_1, image, chat_2, image, hd_num=hd_num)
print("compare_res: ", compare_res)
# >>> compare_res: True
# rank multiple chats, it will return the ranking index of each chat
# the chat with the highest score will have ranking index as 0
with torch.autocast(device_type='cuda', dtype=torch.float16):
rank_res = model.rank([chat_1, chat_2], [image, image], hd_num=hd_num)
print("rank_res: ", rank_res) # lower index means higher score
# >>> rank_res: [0, 1]
I met the same problem. Have you solved this problem?
I met the same problem. Have you solved this problem?
I met the same problem. Have you solved this problem?
I met the same problem. Have you solved this problem?
I have resolved this problem. You should change the visions of transformer and other tools. Detailed description is on
pip install transformers==4.33.2 timm==0.4.12 sentencepiece==0.1.99 gradio==4.13.0 markdown2==2.4.10 xlsxwriter==3.1.2 einops