OpenCL-CTS icon indicating copy to clipboard operation
OpenCL-CTS copied to clipboard

Negative testing for command-buffers

Open EwanC opened this issue 2 years ago • 1 comments

The https://github.com/KhronosGroup/OpenCL-CTS/issues/1369 test plan for cl_khr_command_buffer doesn't include any negative testing. However, there are many error codes defined by the extension, it would be nice to verify are reported correctly.

The majority of existing coverage in the CTS does not have negative testing, so this is a request for coverage over and above what we would normally do. However, negative testing does have advantages and we are thinking about adding it in other cases, see https://github.com/KhronosGroup/OpenCL-CTS/issues/516

Note that if https://github.com/KhronosGroup/OpenCL-Docs/pull/885 gets merged it will mean that we won't need to do Invalid state checks.

EwanC avatar Mar 09 '23 12:03 EwanC

Hi, regarding to Command Buffer extension from #the spec there is 18 functions to cover with tests. Total number of error codes is 108. My plan is to group functions by lets say "family" and possibly error codes. In the end number of tests will be smaller than 108. At this moment it will be about 80. Functions will be grouped like:

  • [x] clCreateCommandBufferKHR()

10 error codes

  • [x] clGetCommandBufferInfoKHR()

4 error codes

  • [x] clRetainCommandBufferKHR() clReleaseCommandBufferKHR()

3 error codes

  • [x] clFinalizeCommandBufferKHR()

4 error codes

  • [x] clEnqueueCommandBufferKHR()

13 error codes

  • [x] clCommandNDRangeKernelKHR()

9 error codes

  • [x] clCommandBarrierWithWaitListKHR()

8 error codes

  • [x] clCommandCopyBufferKHR() clCommandCopyBufferRectKHR() clCommandCopyBufferToImageKHR()

6 error codes

  • [x] clCommandCopyImageKHR() clCommandCopyImageToBufferKHR()

6 error codes

  • [x] clCommandFillBufferKHR() clCommandFillImageKHR()

6 error codes

  • [x] clCommandSVMMemcpyKHR() clCommandSVMMemFillKHR()

6 error codes

Thinking about merging clCommandNDRangeKernelKHR() and clCommandBarrierWithWaitListKHR() because error codes for them looks very similar. I will not include tests for error codes like CL_OUT_OF_RESOURCES, CL_OUT_OF_HOST_MEMORY because at this moment it will be hard to simulate such scenario. That's the plan, it might change a little while implementing tests.

kamil-goras-mobica avatar Feb 26 '24 10:02 kamil-goras-mobica

@EwanC Please tell me if there is something to do with this ticket. Looks all planned items are done and merged.

kamil-goras-mobica avatar Jun 11 '24 15:06 kamil-goras-mobica

@EwanC Please tell me if there is something to do with this ticket. Looks all planned items are done and merged.

I don't think there's anything else to do now, and if something else is identified later it can be created a new ticket. Thanks for your work, I'll close this.

EwanC avatar Jun 11 '24 15:06 EwanC