hyperparameter tuning

[The-Ineffable-Alias]

is optuna and the like a good fit for bertopic?

[MaartenGr]

The difficulty with hyperparameter tuning is that you typically need a ground truth and an objective evaluation metric for it to properly work. Due to the somewhat subjective nature of topic modeling, there usually are no ground truths to be found or evaluation metrics without their flaws. So while Optuna is a great framework for hyperparameter tuning, I personally would not rely solely on such a framework for hyperparameter tuning since human evaluation is almost a necessity.

Especially with topic modeling approaches, it is important to get a feeling of intuition with the model. In practice, there is actually a good chance that optimizing for a specific evaluation metric, like coherence or diversity will not necessarily result in the best model. I would advise tuning the parameters yourself until you find a set that works for your specific use case.