Zach Mueller

Hmm okay I'll try giving it a look tommorow.

Subject: Text

FSDP support for fp8 is experimental and is on NVIDIA's roadmap (with currently no public prototype yet). We need to wait on them.

Use `accelerator.utils.gather`. We definitely could add this to the docs example.

Something like so: ```python # Assume there are two processes from accelerate import PartialState from accelerate.utils import gather from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline model = AutoModelForTokenClassification.from_pretrained('ner') tokenizer = AutoTokenizer.from_pretrained('ner')...

They should be tensors, ideally. Otherwise use the gather_object function instead

You’re more than welcome to add it to the distributed inference tutorial, however it is **strongly** encouraged to work with tensors up until after calling gather() as gather_object does not...

@iantbutler01 the docs need to be updated here, as we decided to go with the plug-in rather than as a param to the `__init__` (to not have too many parameters)....

@iantbutler01 good to close this issue? :)

cc @SunMarc :)