[Good First Issue]: Verify dolly-v2-3b with GenAI text_generation

[p-wysocki]

Context

This task regards enabling tests for dolly-v2-3b. You can find more details under openvino_notebooks LLM question answering README.md.

Please ask general questions in the main issue at https://github.com/openvinotoolkit/openvino.genai/issues/259

What needs to be done?

Described in the main Discussion issue at: https://github.com/openvinotoolkit/openvino.genai/issues/259

Example Pull Requests

Described in the main Discussion issue at: https://github.com/openvinotoolkit/openvino.genai/issues/259

Resources

• Contribution guide - start here!
• Intel DevHub Discord channel - engage in discussions, ask questions and talk to OpenVINO developers

Contact points

Described in the main Discussion issue at: https://github.com/openvinotoolkit/openvino.genai/issues/259

Ticket

No response

[Vishwa44]

.take

[Vishwa44]

Hi @Wovchena can this task be assigned to me

[p-wysocki]

Hello @Vishwa44, we enabled the take command in GenAI repo just after you wrote it. :)

I assigned you the task, thanks for taking a look! Please let us know if you have any questions.

[Vishwa44]

@p-wysocki I just got done testing the model. It's working as expected I'll raise a pr adding it to the supported model list

[p-wysocki]

Perfect! Please make sure to link it to the issue. :)

[Vishwa44]

Hi @p-wysocki, this is the link to my pr verifed-dolly

[p-wysocki]

Hello @Vishwa44, thank you for your contribution! Could you please add tests? You can check other completed issues for reference or the issue description at https://github.com/openvinotoolkit/openvino.genai/issues/259.

[Vishwa44]

Hi @p-wysocki I'm really sorry but I'm new to this, can you share some handy guide on how to add tests for github actions runner, I'll make the required changes and get it done asap

[mlukasze]

@Wovchena || @pavel-esir : could you please close a ticket, both PRs has been merged. Also, please add 2024.1 as a milestone to this ticket. Thanks.

[pavel-esir]

@mlukasze those prs were for Readme changes, but there is a left test for dolly 2-3b. Let's close after the tests are implemented. @Vishwa44 sorry for late reply, you can take as example already existing test where we compare output generated by us with HF transformers output, e.g.: https://github.com/openvinotoolkit/openvino.genai/blob/master/.github/workflows/causal_lm_cpp.yml#L151-L168

you can simplify code above and compare only for greedy scenario, run greedy_causal_lm instead of beam_search_causal_lm and generate in HF with much shorter command generate(**tokenized, max_length=100, do_sample=False)

Please let me know if you have any questions

[mlukasze]

got it, thanks for clarification :)

[Vishwa44]

@pavel-esir I'll raise the pr asap for adding the test, thank you

[andrei-kochin]

@Vishwa44 are you still working on this? there is more than month without an update

[p-wysocki]

The model still needs to be validated and the task is now open again. @Vishwa44 if you're still working on this please let us know, for now I reopened the task.

[Vishwa44]

@p-wysocki sorry for the late reply, I'll resume on this and finish this task

[Apoorv012]

@p-wysocki Is this issue up for grab?

[Apoorv012]

.take

[github-actions[bot]]

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

[kshitij01042002]

.take

[github-actions[bot]]

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

[mlukasze]

hey @kshitij01042002 any help needed?

[Manideep-Maddileti]

.take

[github-actions[bot]]

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

[Pranjal0901]

.take

[github-actions[bot]]

Thanks for being interested in this issue. It looks like this ticket is already assigned to a contributor. Please communicate with the assigned contributor to confirm the status of the issue.