We'd like your feedback on our MLflow experience!

[AbeOmor]

Would love if you could take sometime to share your experience with the MLflow team.

In the past year, MLflow has released multiple features notably MLflow evaluate, Prompt UI, MLflow datasets, new chart experience, and more, and the MLflow product team would love to hear your thoughts on some of the new features and how you envision MLflow growing to meet your needs, especially with Deep Learning or LLMs.

The survey would take 3-5 mins of your time and would greatly help the MLflow team in designing the next chapter of the product.

Fill out the survey HERE

Feel free to add more feedback in the comments below

[Jamesetay1]

I have just recently started using MLFLow for Prompt Engineering and interacting with the gateway through a notebook has been really seamless. One feature in the Prompt UI I'd like to see (or perhaps don't understand) is that when you add a new row to or duplicate a run it automatically populates the variables for you. I only want to change one of the variables of the three for a new run, but currently I have to copy and paste all of them from outside the UI.

[github-actions[bot]]

@mlflow/mlflow-team Please assign a maintainer and start triaging this issue.

[e-dorigatti]

Please improve the responsiveness of the web UI! Performance issues were raised several times in the past (#627, #925, #1517, #1571, #1763, #1902, #3334, #4034, #4305, #4454, #5653) and are still not fixed. It only takes a few hundreds runs to bring the web UI to its knees. At this point I am barely able to browse the results of a single hyperparameter search, and I need to start looking for alternative products.

[rorour]

I'm new to MLFlow and have some questions/suggestions related to logging/viewing metrics.

1. There doesn't seem to be anywhere in the UI that displays the number of data points that have been logged for a given metric. That would be useful information, especially on the plot view.

2. Related, on the Overview of a run, the Metrics section doesn't show any indication of whether a metric has logged multiple values - you have to click through to the plot view to discover this. In this view it's also not clear whether the singular value shown is the latest logged value, or the last step (which could be different if logged out of order), or the max, etc. It would be much more informative to have detail on min/max/latest/total num data points in this view.


3. Is there a way to reduce overhead by logging multiple steps of the same metric with one API call? l see that log_metrics allows to log one step for multiple metrics at once, but I would like to log, say, 1000 steps for the "loss" metric at once. Making an API call in every epoch is expensive.

I'm happy to make separate issues for these if needed.

[wzm2256]

Please improve the responsiveness of the web UI! Performance issues were raised several times in the past (#627, #925, #1517, #1571, #1763, #1902, #3334, #4034, #4305, #4454, #5653) and are still not fixed. It only takes a few hundreds runs to bring the web UI to its knees. At this point I am barely able to browse the results of a single hyperparameter search, and I need to start looking for alternative products.

@e-dorigatti Hi, did you find any other products? I am also looking for an alternative. The performance is still a problem by the time I write this reply, but it seems the MLFlow team does not have time to fix this, because there are 1K+ opened issues, and many of the closed issues are actually also not fixed.

[e-dorigatti]

Please improve the responsiveness of the web UI! Performance issues were raised several times in the past (#627, #925, #1517, #1571, #1763, #1902, #3334, #4034, #4305, #4454, #5653) and are still not fixed. It only takes a few hundreds runs to bring the web UI to its knees. At this point I am barely able to browse the results of a single hyperparameter search, and I need to start looking for alternative products.

@e-dorigatti Hi, did you find any other products? I am also looking for an alternative. The performance is still a problem by the time I write this reply, but it seems the MLFlow team does not have time to fix this, because there are 1K+ opened issues, and many of the closed issues are actually also not fixed.

Hi @wzm2256, since then we have switched to tensorboard and found no particular issues in the process.