DeepRule icon indicating copy to clipboard operation
DeepRule copied to clipboard

Recreate numbers in table 2

Open crazysal opened this issue 3 years ago • 3 comments

Hi, is there an evaluation script to re-create numbers of table 2 from paper.

The test_pipe_type_cloud.py only gives json as output. Where are the eval metrics ?

crazysal avatar Aug 16 '21 21:08 crazysal

This would be really helpful. Used pre-trained models to benchmark on https://chartinfo.github.io/
the performance score is 0.34 (task 6b) for line charts specifically where their and your metrics described in the paper match exactly.

crazysal avatar Aug 31 '21 13:08 crazysal

@crazysal how about the score of task 6a

xuewenyuan avatar Apr 22 '23 06:04 xuewenyuan

out of box is 0.73

crazysal avatar Jul 04 '23 17:07 crazysal