Adriane Boyd

Preserve `tokenizer.is_fast` as `use_fast` in `HFShim` in order to be able to restore the same slow/fast tokenizer settings. This will be necessary when fast tokenizers switch to offset alignments from...

A bit brute-force on the exception handling, but I think this function should never crash if it all possible.

Replace Pipe with Callable in Language

3

## Description Replace `Pipe` with `Callable` in `Language`. ### Types of change ? ## Checklist - [x] I confirm that I have the right to submit this contribution under the...

Replace EntityRuler with SpanRuler implementation

4

## Description Remove `EntityRuler` and rename the `SpanRuler`-based `future_entity_ruler` to `entity_ruler`. Main changes: * It is no longer possible to load patterns on init as with `EntityRuler(patterns=)`. * The older...

Remove default stop words

7

## Description Remove default stop words Stop words are task-specific and attempting to maintain "general-purpose" stop word lists for many different languages is not feasible. None of the underlying functionality...

Refactor pipe(as_tuples) into a separate method

1

## Description Refactor pipe(as_tuples) into a separate method ### Types of change ? ## Checklist - [x] I confirm that I have the right to submit this contribution under the...

Update to use setup.cfg

3

Update to use `setup.cfg`

### System Info - `transformers` version: 4.28.1 - PyTorch version (GPU?): 2.0.0+cu117 (True) ### Who can help? @Narsil ### Information - [ ] The official example scripts - [ ]...

## Description The use of `build-constraints.txt` is an explosion-internal design intertwined with `wheelwright`. After dropping python 3.8, the `oldest-supported-numpy`-ish build constraints are no longer necessary, which greatly simplifies the `cibuildwheel`...