ray
ray copied to clipboard
TypeError: type of val must be one of (Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float..
I was trying to run the toy example to tune hyper parameters with Ray Tune and Ax Search in the goggle colab. I expected to get the best hyper parameters but but I am encountering the following error:
[INFO 05-25 07:01:50] ax.service.ax_client: Generated new trial 2 with parameters {'x1': 0.451661, 'x2': 0.981255, 'x3': 0.719248, 'x4': 0.905622, 'x5': 0.3932, 'x6': 0.41869, 'iterations': 100}.
2023-05-25 07:01:59,804 ERROR trial_runner.py:671 -- Trial easy_objective_f6fdb8ff: Error stopping trial.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ax/utils/common/typeutils.py", line 86, in checked_cast_complex
check_type("val", val, typ)
File "/usr/local/lib/python3.10/dist-packages/typeguard/__init__.py", line 757, in check_type
checker_func(argname, value, expected_type, memo)
File "/usr/local/lib/python3.10/dist-packages/typeguard/__init__.py", line 558, in check_union
raise TypeError('type of {} must be one of ({}); got {} instead'.
TypeError: type of val must be one of (Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]], float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]], List[Tuple[Dict[str, Union[str, bool, float, int, NoneType]], Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]]]], List[Tuple[Dict[str, Hashable], Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]]]]); got dict instead
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ray/tune/execution/trial_runner.py", line 662, in stop_trial
self._search_alg.on_trial_complete(
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/search_generator.py", line 142, in on_trial_complete
self.searcher.on_trial_complete(trial_id=trial_id, result=result, error=error)
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/concurrency_limiter.py", line 138, in on_trial_complete
self.searcher.on_trial_complete(trial_id, result=result, error=error)
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/ax/ax_search.py", line 323, in on_trial_complete
self._process_result(trial_id, result)
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/ax/ax_search.py", line 343, in _process_result
self._ax.complete_trial(trial_index=ax_trial_index, raw_data=metric_dict)
File "/usr/local/lib/python3.10/dist-packages/ax/service/ax_client.py", line 759, in complete_trial
data_update_repr = self._update_trial_with_raw_data(
File "/usr/local/lib/python3.10/dist-packages/ax/service/ax_client.py", line 1556, in _update_trial_with_raw_data
update_info = trial.update_trial_data(
File "/usr/local/lib/python3.10/dist-packages/ax/core/trial.py", line 298, in update_trial_data
arm_name: checked_cast_complex(
File "/usr/local/lib/python3.10/dist-packages/ax/utils/common/typeutils.py", line 89, in checked_cast_complex
raise ValueError(message or f"Value was not of type {typ}: {val}")
ValueError: Raw data must be data for a single arm for non batched trials.
2023-05-25 07:02:00,444 ERROR trial_runner.py:671 -- Trial easy_objective_ab9d36df: Error stopping trial.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ax/utils/common/typeutils.py", line 86, in checked_cast_complex
check_type("val", val, typ)
File "/usr/local/lib/python3.10/dist-packages/typeguard/__init__.py", line 757, in check_type
checker_func(argname, value, expected_type, memo)
File "/usr/local/lib/python3.10/dist-packages/typeguard/__init__.py", line 558, in check_union
raise TypeError('type of {} must be one of ({}); got {} instead'.
TypeError: type of val must be one of (Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]], float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]], List[Tuple[Dict[str, Union[str, bool, float, int, NoneType]], Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]]]], List[Tuple[Dict[str, Hashable], Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]]]]); got dict instead
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ray/tune/execution/trial_runner.py", line 662, in stop_trial
self._search_alg.on_trial_complete(
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/search_generator.py", line 142, in on_trial_complete
self.searcher.on_trial_complete(trial_id=trial_id, result=result, error=error)
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/concurrency_limiter.py", line 138, in on_trial_complete
self.searcher.on_trial_complete(trial_id, result=result, error=error)
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/ax/ax_search.py", line 323, in on_trial_complete
self._process_result(trial_id, result)
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/ax/ax_search.py", line 343, in _process_result
self._ax.complete_trial(trial_index=ax_trial_index, raw_data=metric_dict)
File "/usr/local/lib/python3.10/dist-packages/ax/service/ax_client.py", line 759, in complete_trial
data_update_repr = self._update_trial_with_raw_data(
File "/usr/local/lib/python3.10/dist-packages/ax/service/ax_client.py", line 1556, in _update_trial_with_raw_data
update_info = trial.update_trial_data(
File "/usr/local/lib/python3.10/dist-packages/ax/core/trial.py", line 298, in update_trial_data
arm_name: checked_cast_complex(
File "/usr/local/lib/python3.10/dist-packages/ax/utils/common/typeutils.py", line 89, in checked_cast_complex
raise ValueError(message or f"Value was not of type {typ}: {val}")
ValueError: Raw data must be data for a single arm for non batched trials.
[INFO 05-25 07:02:02] ax.service.ax_client: Generated new trial 3 with parameters {'x1': 0.066228, 'x2': 0.189272, 'x3': 0.109959, 'x4': 0.977603, 'x5': 0.338993, 'x6': 0.884127, 'iterations': 100}.
[INFO 05-25 07:02:02] ax.service.ax_client: Generated new trial 4 with parameters {'x1': 0.777873, 'x2': 0.877265, 'x3': 0.403814, 'x4': 0.25485, 'x5': 0.582553, 'x6': 0.564855, 'iterations': 100}.
2023-05-25 07:02:12,987 ERROR trial_runner.py:671 -- Trial easy_objective_24faac17: Error stopping trial.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ax/utils/common/typeutils.py", line 86, in checked_cast_complex
check_type("val", val, typ)
File "/usr/local/lib/python3.10/dist-packages/typeguard/__init__.py", line 757, in check_type
checker_func(argname, value, expected_type, memo)
File "/usr/local/lib/python3.10/dist-packages/typeguard/__init__.py", line 558, in check_union
raise TypeError('type of {} must be one of ({}); got {} instead'.
TypeError: type of val must be one of (Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]], float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]], List[Tuple[Dict[str, Union[str, bool, float, int, NoneType]], Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]]]], List[Tuple[Dict[str, Hashable], Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]]]]); got dict instead
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ray/tune/execution/trial_runner.py", line 662, in stop_trial
self._search_alg.on_trial_complete(
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/search_generator.py", line 142, in on_trial_complete
self.searcher.on_trial_complete(trial_id=trial_id, result=result, error=error)
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/concurrency_limiter.py", line 138, in on_trial_complete
self.searcher.on_trial_complete(trial_id, result=result, error=error)
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/ax/ax_search.py", line 323, in on_trial_complete
self._process_result(trial_id, result)
File "/usr/local/lib/python3.10/dist-packages/ray/tune/search/ax/ax_search.py", line 343, in _process_result
self._ax.complete_trial(trial_index=ax_trial_index, raw_data=metric_dict)
File "/usr/local/lib/python3.10/dist-packages/ax/service/ax_client.py", line 759, in complete_trial
data_update_repr = self._update_trial_with_raw_data(
File "/usr/local/lib/python3.10/dist-packages/ax/service/ax_client.py", line 1556, in _update_trial_with_raw_data
update_info = trial.update_trial_data(
File "/usr/local/lib/python3.10/dist-packages/ax/core/trial.py", line 298, in update_trial_data
arm_name: checked_cast_complex(
File "/usr/local/lib/python3.10/dist-packages/ax/utils/common/typeutils.py", line 89, in checked_cast_complex
raise ValueError(message or f"Value was not of type {typ}: {val}")
ValueError: Raw data must be data for a single arm for non batched trials.
2023-05-25 07:02:13,233 ERROR trial_runner.py:671 -- Trial easy_objective_d2656154: Error stopping trial.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ax/utils/common/typeutils.py", line 86, in checked_cast_complex
check_type("val", val, typ)
File "/usr/local/lib/python3.10/dist-packages/typeguard/__init__.py", line 757, in check_type
checker_func(argname, value, expected_type, memo)
File "/usr/local/lib/python3.10/dist-packages/typeguard/__init__.py", line 558, in check_union
raise TypeError('type of {} must be one of ({}); got {} instead'.
TypeError: type of val must be one of (Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]], float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]], List[Tuple[Dict[str, Union[str, bool, float, int, NoneType]], Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]]]], List[Tuple[Dict[str, Hashable], Dict[str, Union[float, numpy.floating, numpy.integer, Tuple[Union[float, numpy.floating, numpy.integer], Union[float, numpy.floating, numpy.integer, NoneType]]]]]]); got dict instead
Versions / Dependencies
Ray version: 2.4.0 Ax version: 0.3.3.dev18 python: 3.10
Reproduction script
I was trying to run the toy example given in the documentation: https://docs.ray.io/en/latest/tune/examples/includes/ax_example.html
What is the issue here? Any help or guidance, please?
Issue Severity
High: It blocks me from completing my task.
Hi,
Any update on this issue?
Hi @asifmustafa87,
I tested this on py38 and it seems to work with ax-platform==0.3.2
and typeguard==2.13.3
.
Using py310 which is the default on Google Colab doesn't work for some reason. Generally, this problem would be resolved by this PR on Ax's side: https://github.com/facebook/Ax/pull/1510
For now, I recommend just lowering the python version and seeing if that works for you.
Hi, I am not using this anymore. you can close this issue! Thank you for your time and help!