TEXTOIR icon indicating copy to clipboard operation
TEXTOIR copied to clipboard

Error in reproduction of run_DA-ADB.sh results

Open EdwardBurgin opened this issue 1 year ago • 2 comments

Is the example found in open_intent_detection/examples/run_DA-ADB.sh meant to reproduce the published results? Running this and comparing to ADB gives significantly worse results. I took a look to see why this is briefly but didn't find an obvious answer.

The average of ADB across the experiments gave 84% and DA-ADB gave 47%

Example of ADB results from which averages of Acc where calculated DA-ADB_RES

The only change was the location of the bert model in hyper-parameters which was the same for both methods:
'bert_model': "bert-base-uncased", Instead of: 'bert_model': "/home/sharing/disk1/pretrained_embedding/bert/uncased_L-12_H-768_A-12/",

Thanks for your help!

EdwardBurgin avatar Jan 03 '24 16:01 EdwardBurgin