FEAT replace fastchat in GCG

[romanlutz]

Is your feature request related to a problem? Please describe.

fastchat hasn't had releases for 15 months and we rely on it to provide conversation templates. This task is about investigating alternatives since we use relatively little of their functionality:

from fastchat.conversation import Conversation, SeparatorStyle
from fastchat.model import get_conversation_template

and

    for template in params.conversation_templates:
        if template in ["llama-2", "mistral", "llama-3-8b", "vicuna"]:
            raw_conv_templates.append(get_conversation_template(template)),
        elif template in ["phi-3-mini"]:
            conv_template = Conversation(
                name="phi-3-mini",
                system_template="<|system|>\n{system_message}",
                system_message="",
                roles=("<|user|>", "<|assistant|>"),
                sep_style=SeparatorStyle.CHATML,
                sep="<|end|>",
                stop_token_ids=[32000, 32001, 32007],
            )
            raw_conv_templates.append(conv_template)
        else:
            raise ValueError("Conversation template not recognized")

We should be able to expand to more models. Currently, we can work with

"mistral", "llama_2", "llama_3", "vicuna", "phi_3_mini", "all_models"

where the last one means all of the before.

I would like a much expanded set of models. We need to explore how that can be facilitated. Perhaps there are up-to-date packages now that make this a lot easier.

[zhang-shubai]

Cannot agrree more! — the FastChat library definitely needs to be replaced.

It provides conversation templates for different LLMs, which GCG find where the adversial suffixs are and which tokens to replace. However, it hasn't been updated in a long time, so it lacks support for many newer models, such as LLaMA-3, Qwen-2.5, and others. I’ve had to manually write these conversation templates myself, which is quite time-consuming.

Additionally, FastChat depends on outdated versions of transformers or torch, which can easily lead to version conflicts.

[romanlutz]

Indeed! Have you come across options?