prompteus

Any updates?

I need this too. I think there should be these two methods: `String getSearchQuery()` `void submit()`

Any updates on merging this? @tiangolo

just in addition, I believe the simplest PEFT setup is to just freeze most of the model and let the optimizer update only some parameters - for example, BitFit https://arxiv.org/abs/2106.10199...

@ontocord Yes, I agree, part of the tasks inside promptsource / natural instructions are academic, but it would still teach the model to 1) understand instructions and 2) pick up...

@ontocord I'm currently having an exam period, so I'm unavailable for the next 3 weeks, but I'd be happy to help with it afterward. In the meantime, I will think...

Just found this doc by Yao Fu at Allen AI about the evolution of OpenAI models from GPT-3 to now: [How does GPT Obtain its Ability? Tracing Emergent Abilities of...

Well, I ended up doing the same, it's the easiest workaround, but the binding would be much cleaner.

Generation is not even needed when comparing several options. One can do a single forward pass for each option, take all next-token logprobs, and combine them into the probability of...

What @slundberg suggests seems to me like a good workaround. Still, I am happy to hear that there are plans to support log_prob calculation as well.