FEDOT
FEDOT copied to clipboard
Handle with `MemoryError` in `_raise_evaluating_exception()`
Launched Fedot with data including large categories features. Fedot tryied applying one-hot encoding at preprocessing, but it crashed with MemoryError
. It include raise ValueError
message with noninformative fit_failed_info = '.'
in traceback.
In my opinion, it is not obvious for user to use safe_mode=True
in API after this message, and my suggestion is to handle this type error and add informative advice description.
Traceback (most recent call last):
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\api\api_utils\assumptions\assumptions_handler.py", line 70, in fit_assumption_and_check_correctness
pipeline.fit(data_train, n_jobs=eval_n_jobs)
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\core\pipelines\pipeline.py", line 187, in fit
copied_input_data = self._preprocess(input_data)
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\core\pipelines\pipeline.py", line 139, in _preprocess
copied_input_data = self.preprocessor.optional_prepare_for_fit(pipeline=self,
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\preprocessing\preprocessing.py", line 140, in optional_prepare_for_fit
self._prepare_optional(pipeline, data, DEFAULT_SOURCE_NAME)
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\preprocessing\preprocessing.py", line 284, in _prepare_optional
data = action_if_no_tag(data, source_name)
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\preprocessing\preprocessing.py", line 415, in _apply_categorical_encoding
output_data = encoder.transform_for_fit(data)
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\core\operations\evaluation\operation_implementations\implementation_interfaces.py", line 45, in transform_for_fit
return self.transform(input_data)
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\core\operations\evaluation\operation_implementations\data_operations\categorical_encoders.py", line 64, in transform
transformed_features = self._apply_one_hot_encoding(features)
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\core\operations\evaluation\operation_implementations\data_operations\categorical_encoders.py", line 103, in _apply_one_hot_encoding
transformed_features = np.hstack(frames)
File "<__array_function__ internals>", line 180, in hstack
File "C:\Users\andre\Documents\GitHub\FEDOT\venv\lib\site-packages\numpy\core\shape_base.py", line 345, in hstack
return _nx.concatenate(arrs, 1)
File "<__array_function__ internals>", line 180, in concatenate
MemoryError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\andre\AppData\Local\Programs\Python\Python310\lib\contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\api\time.py", line 72, in launch_assumption_fit
yield
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\api\api_utils\api_composer.py", line 231, in compose_fedot_model
assumption_handler.fit_assumption_and_check_correctness(initial_assumption[0],
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\api\api_utils\assumptions\assumptions_handler.py", line 85, in fit_assumption_and_check_correctness
self._raise_evaluating_exception(ex)
File "C:\Users\andre\Documents\GitHub\FEDOT\fedot\api\api_utils\assumptions\assumptions_handler.py", line 93, in _raise_evaluating_exception
raise ValueError(advice_info)
ValueError: Initial pipeline fit was failed due to: . Check pipeline structure and the correctness of the data
Process finished with exit code 1