张磊

from tqdm import tqdm for prompt in prompts: preds = [] labels = [] for data in tqdm(dataset): # process input input_text = pb.InputProcess.basic_format(prompt, data) label = data['label'] print(type(input_text)) raw_pred...

可以添加一个新的攻击方法吗？https://github.com/AdvAttack/BU-SPO

1

因为想要使用https://github.com/AdvAttack/BU-SPO这个攻击方法去进行一个大模型的攻击，可以将这个写进来吗？如果要自己写的话，是否有很好的模块添加的路线？

Why are the experimental results different?

3

## bertattack original prompt: Evaluate the sentiment of the given text and classify it as 'positive' or 'negative': original score: 0.4934426229508197 attacked prompt: Evaluate the sеntiment of the given text...

Because the accuracy of my experimental results does not match the paper. Because I want to directly evaluate the prompt after being attacked by text to see if the score...

Want to know what code is used to implement the attention visualization in the figure below?

2


In order to be able to read the local GLUE data set, I modified the GLUE code so that the attack evaluation can be carried out, but the current score of the output is always 0. I want to know why?

3

class GLUE(Dataset): """ GLUE class is a dataset class for the General Language Understanding Evaluation benchmark, supporting multiple natural language understanding tasks. Examples: [{'content': "it 's a charming and often...

May I ask how you used a large generative model to allow him to accurately answer classification tasks for experiments?

3

I want to test other models according to your method, but I find it difficult to classify generative models. Please tell me how you solved it?

The MMLU dataset results of google_flan_t5_large are lower than your experimental results

4

 ```python ```# create dataset dataset = pb.DatasetLoader.load_dataset("mmlu",local_path=localpathconfig.MMLU_PATH) #还有mrpc的实验 logging.info(f"数据集: mmlu") # 确保数据集包含足够的数据 if len(dataset) >= 1000: # 选择前1000条记录 validation_dataset = dataset[:1000] else: validation_dataset=dataset prompts = [ "In relation to...

If I don't change the @torch.no_grad() in the code, I can't run it and get an error that the loss forward propagation cannot be done. If I comment out @torch.no_grad(),...

I want to find where I can jump to this

1

def get_indices_to_order(self, current_text, **kwargs): """Applies ``pre_transformation_constraints`` to ``text`` to get all the indices that can be used to search and order. Args: current_text: The current ``AttackedText`` for which we need...