Nicolas Patry

> `audio-token-classification`? scream You're actually pretty on spot on IMO, since `token-classification` is actually `text-segmentation` I think. It's also aligned with `image-segmentation`. Which basically should be a list of "objects"...

`speech-segmentation` was never deprecated, but it also never had widget support afaik. It's output is not `audio` so I don't see how `audio-to-audio` could be used: https://github.com/huggingface/huggingface_hub/blob/main/api-inference-community/docker_images/superb/app/pipelines/speech_segmentation.py

I think we can keep the PR as is, merge it when ready, so things are functional (even though less than perfect). And when support for `audio-segmentation` is ready (or...

I think it fills `fill-mask` quite nicely, in the sense the given a masked input, the model should tell us what should be under mask. Now potential caveats/pains: - Currently...

You could try and create inverse scripts for the conversion you found. But it's not going to be trivial. You need to create the protobuf sentencepiece expects. Not sure I...

Awesome. Do you mind explaining a little more or giving links for potential readers that would want to do the same?

Everything you need is here: https://github.com/huggingface/transformers/blob/main/src/transformers/convert_slow_tokenizer.py There is no simple tutorial, there are many configurations in `tokenizers` that could achieve what you want, with various tradeoffs. What I recommend is...

That would be nice, but requires pretty much changing `generate` upside down and inside out. This is what we have done here: https://github.com/huggingface/text-generation-inference which was required to get max performance...

Pipeline is stateless, so it cannot keep the `past_key_values` and for you to send it again and again kind of defeats the purpose of a pipeline imo (since you can't...

@OlivierDehaene Tagging just because we were talking about the stream process in `text-generation-inference` :)