My device can not run this model, it tip 'Killed'

[xiashuqin89]

trafficstars

[mi4uu]

try to run it like that:

pipe=pipeline(
            "text-generation",
            model="bigcode/starcoder",
            torch_dtype=torch.bfloat16,
    
            device_map='auto',
        )
        
generation_config = GenerationConfig.from_pretrained(checkpoint)
response = pipe('def main('    , generation_config= GenerationConfig.from_dict(
            {**generation_config.to_dict(), 'max_new_tokens':max_length}
        ))
text = response[0]['generated_text']
print(f'{text}')

[loubnabnl]

It's probably due to OOM, you can check the hardware requirements at: https://github.com/bigcode-project/starcoder#inference-hardware-requirements