Wei-Lin Chiang
Wei-Lin Chiang
https://github.com/skypilot-org/skypilot/pull/1500 proposes a solution by https://github.com/skypilot-org/skypilot/pull/1500/files#diff-6749e0638b4e0e0bf9e5b2e0be361b6394a82458a729e81c2ff2ca6dcd6a653aR315 to explicitly terminate the cluster before launching another one as `status -r` may not work.
Fixed by https://github.com/skypilot-org/skypilot/pull/1500.
Ah okay I think another issue I hit is I'm testing with a much smaller spot controller (2 CPUs). When all the spot smoke test submitted, some of them will...
We may need to update such instance based info in our catalog? @WoosukKwon wdyt
Ah yes it's duplicated to https://github.com/skypilot-org/skypilot/issues/1006. > One possible solution is to set the Price values of those instances as null, so that they are filtered out. so the question...
Yes, I'm adding the list support now and will update the PR soon. This will require some changes in our optimizer logic. It'd be nice if @WoosukKwon can help to...
As mentioned in our blog [post](https://vicuna.lmsys.org/): > we devised eight question categories, such as Fermi problems, roleplay scenarios, and coding/math tasks, to test various aspects of a chatbot’s performance. The...
Closing this as it's stalled.
Hey @ethanyanjiali great question! do you have any pointer or related reference on how people do such evaluation? is there public dataset?
A colab demo for FastChat will be great. we welcome contribution.