Is using an Instruct model (Mixtral:8x7-INstruct) is relevant with PaperQA ?

[Snikch63200]

Hello,

I'm using Mixtral 8x7b Instruct with PaperQA but but I noticed that the quality of the answers is relatively random (sometimes good, sometimes very bad...).

I read instruct models may have some limitations like :

• context usage
• JSON formatting
• global understanding of complex question

and needs a specific prompt formatting.

So I wonder if an Instruct model is relevant with PaperQA usage and if there is a good way to use it (especially in prompt formatting).

Best regards. @Dosu

[dosubot[bot]]

The use of Instruct models like Mixtral 8x7b with PaperQA can indeed present challenges, particularly in areas such as context usage, JSON formatting, and understanding complex questions. While I couldn't find specific details on the role of Mixtral 8x7b within PaperQA, it's important to note that PaperQA processes JSON outputs from Instruct models using a function designed to handle JSON formatting issues ^[1].

For prompt formatting, while specific guidelines weren't found, it's generally beneficial to ensure that prompts are clear and structured to help the model understand the context better. If you experience inconsistent answer quality, experimenting with different prompt styles and ensuring the model's capabilities align with your needs might help improve results.

To continue talking to Dosu, mention @dosu.