Support testlib SPJ for grader

[Mr-Python-in-China]

[Mr-Python-in-China]

问题来了 test 怎么写

[weilycoder]

在 Windows 下的一个 Bug：

总之重定向到 stdout 的行为没有成功，大概是因为 con 只是输出到控制台。

[Mr-Python-in-China]

在 Windows 下的一个 Bug：

总之重定向到 stdout 的行为没有成功，大概是因为 con 只是输出到控制台。

windows 怎么这么不牛 只能走临时文件了

[Mr-Python-in-China]

@weilycoder 没环境 你再试试

[weilycoder]

我认为我有必要先描述我的测试方法，以便后续讨论：

• 编译后为 a.exe：

  #include <iostream>
  using namespace std;
  
  int main() {
    int a, b;
    cin >> a >> b;
    cout << a + b << endl;
    return 0;
  }
  

• 编译后为 checker.exe：

  #include "testlib.h"
  
  int main(int argc, char *argv[]) {
    registerTestlibCmd(argc, argv);
    int a = inf.readInt(), b = inf.readInt();
    int ret = ouf.readInt();
    if (a + b != ret)
      quitf(_wa, "Expected %d, found %d", a + b, ret);
    quitf(_ok, "Correct");
  }
  

• data.in：

  1 2
  

• test.py：

  # test.py
  from cyaron import *
  from cyaron.graders.testlib_checker import TestlibChecker
  
  tc = TestlibChecker("checker.exe")
  
  Compare.program("a.exe", input="data.in", std_program="a.exe", grader=tc)
  

[weilycoder]

那么，现在有一个奇怪的行为：

https://github.com/luogu-dev/cyaron/blob/f82ae6498f236e65770c0c6d05f938d6b8a2f8d1/cyaron/graders/testlib_checker.py#L33-L36

如果直接跑测试，报错（临时文件名可能不同）：

Traceback (most recent call last):
  File "d:/Admin/Desktop/IDE/github/cyaron/test.py", line 10, in <module>
    Compare.program("a.exe", input="data.in", std_program="a.exe", grader=tc)
  File "d:\Admin\Desktop\IDE\github\cyaron\cyaron\compare.py", line 231, in program
    do(program)
  File "d:\Admin\Desktop\IDE\github\cyaron\cyaron\compare.py", line 225, in do
    cls.__compare_two(program_name, content, std, input_text, grader)
  File "d:\Admin\Desktop\IDE\github\cyaron\cyaron\compare.py", line 43, in __compare_two
    raise CompareMismatch(name, info)
cyaron.compare.CompareMismatch: In program: 'a.exe'. presentation-error Output file not found: "C:\Users\Admin\AppData\Local\Temp\tmp9k2bnhmf"

将 34 行加上 delete=False，报错信息变为：

Traceback (most recent call last):
  File "d:/Admin/Desktop/IDE/github/cyaron/test.py", line 10, in <module>
    Compare.program("a.exe", input="data.in", std_program="a.exe", grader=tc)
  File "d:\Admin\Desktop\IDE\github\cyaron\cyaron\compare.py", line 231, in program
    do(program)
  File "d:\Admin\Desktop\IDE\github\cyaron\cyaron\compare.py", line 225, in do
    cls.__compare_two(program_name, content, std, input_text, grader)
  File "d:\Admin\Desktop\IDE\github\cyaron\cyaron\compare.py", line 43, in __compare_two
    raise CompareMismatch(name, info)
cyaron.compare.CompareMismatch: In program: 'a.exe'. fail Answer file not found: "C:\Users\Admin\AppData\Local\Temp\tmpnrl2c9us"

继续在 35 行加上 delete=False，报错信息变为：

Traceback (most recent call last):
  File "d:/Admin/Desktop/IDE/github/cyaron/test.py", line 10, in <module>
    Compare.program("a.exe", input="data.in", std_program="a.exe", grader=tc)
  File "d:\Admin\Desktop\IDE\github\cyaron\cyaron\compare.py", line 231, in program
    do(program)
  File "d:\Admin\Desktop\IDE\github\cyaron\cyaron\compare.py", line 225, in do
    cls.__compare_two(program_name, content, std, input_text, grader)
  File "d:\Admin\Desktop\IDE\github\cyaron\cyaron\compare.py", line 43, in __compare_two
    raise CompareMismatch(name, info)
cyaron.compare.CompareMismatch: In program: 'a.exe'. fail Unexpected end of file - int32 expected (C:\Users\Admin\AppData\Local\Temp\tmpdmqem_79)

在 33 行加上 delete=False，不再报错。

[weilycoder]

然后如果改成这样，行为看起来就很对：

with tempfile.TemporaryDirectory() as checker_output_dir, \
     open(path_join(checker_output_dir, 'input.txt'), 'w', newline='\n') as inf, \
     open(path_join(checker_output_dir, 'output.txt'), 'w', newline='\n') as outf, \
     open(path_join(checker_output_dir, 'answer.txt'), 'w', newline='\n') as ansf:
    ...

并且如果出现错误可能报错信息也更可读（临时文件见名知意）。

[Mr-Python-in-China]

所以是windows在关闭文件之前不会立即写入？？？

[weilycoder]

鬼知道啥行为？

我只是把 delete 设为 True 了，我没有 close。