Llama-3-8B compatibility

[dnhkng]

Can the adapter be retrained to work with Llama-3-8B, which should perform much better than Vincuna-13B?

[czczup]

Hello, our InternViT-6B can be used together with Llama-3-8B, although this size ratio might not be the most reasonable. We recommend pairing the InternViT-6B with a 20B or 30B language model instead.

[dnhkng]

Is this due to the size of the model or its 'intelligence"?

[fyting]

Is there any experimental or theoretical evidence supporting the statement 'pairing the InternViT-6B with a 20B or 30B language model'? @czczup

[czczup]

Thank you for your attention and we will discuss this in a later technical report.