HugNLP icon indicating copy to clipboard operation
HugNLP copied to clipboard

eval metrics返回结果中eval_macro_f1问题

Open xuechaofei opened this issue 9 months ago • 0 comments

1、使用run_extractive_unified_ie.sh启动训练,运行正常,结果如下: ***** eval metrics ***** epoch = 2.0 eval_ = 0.0 eval_loss = 8.7072 eval_macro_f1 = 0.0 eval_micro_f1 = 0.0 eval_num = 16 eval_runtime = 0:00:01.11 eval_samples = 16 eval_samples_per_second = 14.409 eval_steps_per_second = 1.801

2、将hugnlp_runner.py文件中parser.parse_args_into_dataclasses()改为 parser.parse_json_file(‘config.json'),训练结果如下: ***** eval metrics ***** epoch = 2.0 eval_runtime = 0:00:01.05 eval_samples = 16 eval_samples_per_second = 15.134 eval_steps_per_second = 1.892

eval_macro_f1属性不见了。请问是什么原因?

config.json结构如下: { "model_name_or_path": "/chinese-macbert-large", "data_dir": "/dataset", "output_dir": "/output", "seed": 42, "exp_name": "unified-ie-wjn", "max_seq_length": 512, "max_eval_seq_length": 512, "do_train": true, "do_eval": true, "per_device_train_batch_size": 8, "per_device_eval_batch_size": 8, "gradient_accumulation_steps": 1, "evaluation_strategy": "steps", "learning_rate": 2e-05, "num_train_epochs": 20, "logging_steps": 100000000, "eval_steps": 500, "save_steps": 500, "save_total_limit": 1, "warmup_steps": 200, "load_best_model_at_end": true, "report_to": "none", "task_name": "zh_mrc_instruction", "task_type": "global_pointer", "model_type": "bert", "metric_for_best_model": "macro_f1", "pad_to_max_length": true, "remove_unused_columns": false, "overwrite_output_dir": true, "fp16": true, "label_names": "short_labels", "keep_predict_labels": true, "cache_dir": "/cache" }

xuechaofei avatar May 02 '24 14:05 xuechaofei