Aflah
Hey @BeyonderXX is anyone working on this right now? I'm interested to have a look at this if that works As a starter I feel just parsing and making a...
Yes please this throws off a lot of people!!
@julianatfg Just curious, where did you get the icons from? I also need some for other services like youtube but haven't been able to find one which matches the other...
Same issue on my end
Thanks for sharing the script @merrymercy However, when I start the run the GPU is empty so everything occupying GPU space (which is a single process) must be coming from...
Nope I did not use any other library apart from sglang in the notebook. The first cell I executed contained the code I shared. I'll try setting the `mem_fraction_static`. Maybe...
Running it from the notebook did not work but running it on the terminal and making calls from a notebook does work. Not sure why though.
@merrymercy Thanks for the quick response I just need the log probs of the selected tokens in the sequence given at each position (Option 2). Here is what I do...
@merrymercy Nope I don't need it for the shared prefix, only for the non-shared portions Like for example if the sentences are "Wikipedia originated in India", "Wikipedia originated in U.S.A",...
Yeah I use the normalized logprobs and store them for later analysis. This example looks very relevant. If I understand correctly, something like this would populate the most likely choice...