Effectiveness of APR tools to repair student programs

[ssk1216]

Are the current APR tools effective in fixing the bugs in the programming assignments submitted by the students?
https://www.shinhwei.com/tutoring.pdf paper suggests that repair hints don't seem to be of much use to students. What are the reasons for APR being ineffective when applied in the educational domain?

[monperrus]

I'd say we need to be careful with the wording: "effective" is too vague and too subjective.

The core scientific facts we have is a percentage of repaired student programs for a given benchmark and repair tool.

Both in https://www.shinhwei.com/tutoring.pdf and https://arxiv.org/pdf/1905.11973.pdf (benchmark IntroClassJava), this percentage is approx. 20%.