Update Optuna

[bbudescu]

setup.py specifies that the required version of Optuna to be 2.8.0. However, according to ray docs (check out the second warning on this page) :

Please note that with Optuna 2.10.0 and earlier default MOTPESampler/TPESampler suffer from performance issues when dealing with a large number of completed trials (approx. >100). This will manifest as a delay when suggesting new configurations. This is an Optuna issue and may be fixed in a future Optuna release.

In my own experience, I have also observed the slowdown they mention.

Is there any reason for this version requirement for Optuna? And, if yes, how hard would it be update to the latest version?

[Programmer-RD-AI]

Before transitioning to newer versions, it typically involves a substantial amount of effort. I'm planning to submit a pull request to update Optuna to version 2.10.0, but the rationale behind this decision needs to be verified by Microsoft employees who are actively involved with FLAML.

[thinkall]

Hi @Programmer-RD-AI , as @bbudescu quoted, Optuna 2.10.0 also has the performance issue. I've upgraded Optuna to support latest version in PR #1321 . Could you help review? Thanks.