Sklearn pipeline compatibility and pandas dependencies

[PaulWestenthanner]

Expected Behavior

full compatibility with sklearn pipelines

Actual Behavior

we're only compatible with pandas mode of sklearn. By default a multi-step pipeline, that has an encoder not as first step, e.g.

Pipeline(
    steps=[
        ("preprocessor", SomePreprocessor().set_output("pandas"),
        ("encoder", SomeEncoder()),
    ]
)

will fail if the user does not manually specify set_output('pandas') or configure pandas mode globally for sklearn via sklearn.set_config(transform_output="pandas").

This is not very nice and might lead to errors.

Steps to Reproduce the Problem

1. Create a sklearn Pipeline with 1. step a preprocessor, 2. step an encoder
2. call fit_transform on the pipeline. This will raise an error as category encoders works with dataframes internally and after the first transform and array is given where the column names differ from what the encoder would expect.

Potential Solution

To fix this we'd need to get independent of pandas internally. This is quite difficult and requires some refactoring in all encoders. Mainly how feature names are determined. Also the benefit is rather small since there is an easy workaround in the uncommon case that the encoder is not the first step of a multi-step pipeline.
However, if a major refactoring is done for a potential version 3 we could include this as well.