lshamis

**Describe the bug** The `free_blocks` variable is misused. It seems to be a list[int], but is sometimes used as an int, causing the program to crash. It is correctly used...

[BUG] inference generate raises ValueError

2

**Describe the bug** Calls to `model.generate` with `deepspeed.init_inference` model raises `ValueError: too many values to unpack (expected 2)`. **To Reproduce** ```py import deepspeed import traceback import transformers def main(): pipeline...

Hello, I don't believe there is any way to access logits or the final hidden layer embedding in the current API and would like to request this feature. Thank you