FLAML icon indicating copy to clipboard operation
FLAML copied to clipboard

TextPredictor

Open Qiaochu-Song opened this issue 3 years ago • 1 comments

  • Add an estimator for TextPredictor.
  • Add a test for TextPredictor estimator.

Qiaochu-Song avatar Mar 17 '22 18:03 Qiaochu-Song

Add auto detect multimodal: we should detect multi modal, if the columns contain text field.

To do that, please add an @property function text_columns in DataTransformer, set it to a non-empty list if there exists any text field, None if there does not. Then at this line: https://github.com/microsoft/FLAML/blob/main/flaml/automl.py#L2196, do 3 things:

(1) change self.estimator_list to [multimodalpredictor] if self._transformer.text_columns is non-empty. (2) if self._transformer.text_columns is non-empty, alert users that these column types are set to text by logger.warning, e,g,, "columns type of {} are set to text".format(self._transformer.text_columns) (3) if self._transformer.text_columns is non-empty, and its length = the column number of X_train, assert _is_nlp_task is True because the task must be an NLP task

liususan091219 avatar Apr 01 '22 22:04 liususan091219