[Issue]: PoCL - OpenVX Conformance failures

[kiritigowda]

Problem Description

PoCL is a portable open source (MIT-licensed) implementation of the OpenCL standard. In addition to being an easily portable/layered multi-device open-source OpenCL implementation, a major goal of this project is improving interoperability of diversity of OpenCL-capable devices by integrating them to a single centrally orchestrated platform. Also one of the key goals is to enhance performance portability of OpenCL programs across device types utilizing runtime and compiler techniques.

PoCL Conformance failures

• CPU - Intel CPU
• iGPU - Intel GPU OpenCL driver

[ ======== ]
[ ALL DONE ] 5820 test(s) from 69 test case(s) ran
[ PASSED   ] 5807 test(s)
[ FAILED   ] 13 test(s), listed below:
[ FAILED   ] SmokeTestBase.vxReleaseReferenceBase
[ FAILED   ] SmokeTestBase.vxLoadKernels
[ FAILED   ] SmokeTestBase.vxUnloadKernels
[ FAILED   ] vxConvertDepth.BitExact/6/WRAP/15x17 U8->S16
[ FAILED   ] vxConvertDepth.BitExact/15/WRAP/1x1 S16->U8
[ FAILED   ] GaussianPyramid.GraphProcessing/1/randomInput/VX_BORDER_UNDEFINED/sz=16x16/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.GraphProcessing/3/randomInput/VX_BORDER_UNDEFINED/sz=256x256/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.GraphProcessing/5/randomInput/VX_BORDER_UNDEFINED/sz=640x480/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.GraphProcessing/37/lena/VX_BORDER_UNDEFINED/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.ImmediateProcessing/1/randomInput/VX_BORDER_UNDEFINED/sz=16x16/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.ImmediateProcessing/3/randomInput/VX_BORDER_UNDEFINED/sz=256x256/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.ImmediateProcessing/5/randomInput/VX_BORDER_UNDEFINED/sz=640x480/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.ImmediateProcessing/37/lena/VX_BORDER_UNDEFINED/VX_SCALE_PYRAMID_ORB
[ DISABLED ] 8190 test(s)

Operating System

ANY

CPU

Intel CPU

GPU

Other

Other

Intel iGPU

ROCm Version

ROCm 6.0.0

ROCm Component

MIVisionX

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

[kiritigowda]

@pjaaskel @karolherbst - capturing the current TOT develop issues - Can we reproduce these in PoCL CI?

[pjaaskel]

Added https://github.com/pocl/pocl/issues/1959.

Also please note that also Intel's proprietary CPU driver and GPU driver produce (some of) these issues, thus it's not PoCL only. @karolherbst, what was the end result of the CTS run via Rusticl?

[karolherbst]

@karolherbst, what was the end result of the CTS run via Rusticl?

There is a virtual memory leak (some driver internal reason) and the run aborted at some point, but no fails up to that point.. Need to figure out where it's leaking, but I suspect queue recreation is partly responsible for this.

[karolherbst]

Okay.. I figured out what's going on. Something is leaking cl_event objects, which then keeps the cl_queue object alive internally, which then keeps GPU contexts alive, and we do have some VRAM allocations per context...

Not entirely sure if it's a rusticl internal bug or not, but it's quite unlikely with event objects. I'll try to dig a bit deeper, but it could be that something isn't calling clReleaseEvent often enough.

[karolherbst]

Yeah, I confirmed that there aren't enough clReleaseEvent calls to clean up events returned to the applications (ignoring calls where the pointer us NULL to return them). So it's probably something for you to figure out.

I'll try to mitigate this problem, but so far it seems like an issue inside MIVisionX. Should this be tracked in a new issue?

[karolherbst]

Anyway, with that worked around:

[ ======== ]
[ ALL DONE ] 5820 test(s) from 69 test case(s) ran
[ PASSED   ] 5761 test(s)
[ FAILED   ] 59 test(s), listed below:
[ FAILED   ] SmokeTestBase.vxReleaseReferenceBase
[ FAILED   ] SmokeTestBase.vxLoadKernels
[ FAILED   ] SmokeTestBase.vxUnloadKernels
[ FAILED   ] vxuMultiply.Fuzzy/326/WRAP/TO_ZERO 640x480 S16*S16*(1/2^0)=S16
[ FAILED   ] vxuMultiply.Fuzzy/327/WRAP/TO_ZERO 640x480 S16*S16*(1/2^1)=S16
[ FAILED   ] vxuMultiply.Fuzzy/328/WRAP/TO_ZERO 640x480 S16*S16*(1/2^2)=S16
[ FAILED   ] vxuMultiply.Fuzzy/329/WRAP/TO_ZERO 640x480 S16*S16*(1/2^3)=S16
[ FAILED   ] vxuMultiply.Fuzzy/330/WRAP/TO_ZERO 640x480 S16*S16*(1/2^4)=S16
[ FAILED   ] vxuMultiply.Fuzzy/331/WRAP/TO_ZERO 640x480 S16*S16*(1/2^5)=S16
[ FAILED   ] vxuMultiply.Fuzzy/332/WRAP/TO_ZERO 640x480 S16*S16*(1/2^6)=S16
[ FAILED   ] vxuMultiply.Fuzzy/333/WRAP/TO_ZERO 640x480 S16*S16*(1/2^7)=S16
[ FAILED   ] vxuMultiply.Fuzzy/334/WRAP/TO_ZERO 640x480 S16*S16*(1/2^8)=S16
[ FAILED   ] vxuMultiply.Fuzzy/335/WRAP/TO_ZERO 640x480 S16*S16*(1/2^9)=S16
[ FAILED   ] vxuMultiply.Fuzzy/336/WRAP/TO_ZERO 640x480 S16*S16*(1/2^10)=S16
[ FAILED   ] vxuMultiply.Fuzzy/337/WRAP/TO_ZERO 640x480 S16*S16*(1/2^11)=S16
[ FAILED   ] vxuMultiply.Fuzzy/338/WRAP/TO_ZERO 640x480 S16*S16*(1/2^12)=S16
[ FAILED   ] vxuMultiply.Fuzzy/339/WRAP/TO_ZERO 640x480 S16*S16*(1/2^13)=S16
[ FAILED   ] vxuMultiply.Fuzzy/340/WRAP/TO_ZERO 640x480 S16*S16*(1/2^14)=S16
[ FAILED   ] vxuMultiply.Fuzzy/341/WRAP/TO_ZERO 640x480 S16*S16*(1/2^15)=S16
[ FAILED   ] vxMultiply.Inference/69/WRAP/TO_ZERO S16*S16*(1/2^0)=S16
[ FAILED   ] vxMultiply.Inference/70/WRAP/TO_ZERO S16*S16*(1/2^1)=S16
[ FAILED   ] vxMultiply.Inference/71/WRAP/TO_ZERO S16*S16*(1/2^2)=S16
[ FAILED   ] vxMultiply.Inference/72/WRAP/TO_ZERO S16*S16*(1/2^3)=S16
[ FAILED   ] vxMultiply.Inference/73/WRAP/TO_ZERO S16*S16*(1/2^4)=S16
[ FAILED   ] vxMultiply.Inference/74/WRAP/TO_ZERO S16*S16*(1/2^5)=S16
[ FAILED   ] vxMultiply.Inference/75/WRAP/TO_ZERO S16*S16*(1/2^6)=S16
[ FAILED   ] vxMultiply.Inference/76/WRAP/TO_ZERO S16*S16*(1/2^7)=S16
[ FAILED   ] vxMultiply.Inference/77/WRAP/TO_ZERO S16*S16*(1/2^8)=S16
[ FAILED   ] vxMultiply.Inference/78/WRAP/TO_ZERO S16*S16*(1/2^9)=S16
[ FAILED   ] vxMultiply.Inference/79/WRAP/TO_ZERO S16*S16*(1/2^10)=S16
[ FAILED   ] vxMultiply.Inference/80/WRAP/TO_ZERO S16*S16*(1/2^11)=S16
[ FAILED   ] vxMultiply.Inference/81/WRAP/TO_ZERO S16*S16*(1/2^12)=S16
[ FAILED   ] vxMultiply.Inference/82/WRAP/TO_ZERO S16*S16*(1/2^13)=S16
[ FAILED   ] vxMultiply.Inference/83/WRAP/TO_ZERO S16*S16*(1/2^14)=S16
[ FAILED   ] vxMultiply.Inference/84/WRAP/TO_ZERO S16*S16*(1/2^15)=S16
[ FAILED   ] vxMultiply.Fuzzy/326/WRAP/TO_ZERO 640x480 S16*S16*(1/2^0)=S16
[ FAILED   ] vxMultiply.Fuzzy/327/WRAP/TO_ZERO 640x480 S16*S16*(1/2^1)=S16
[ FAILED   ] vxMultiply.Fuzzy/328/WRAP/TO_ZERO 640x480 S16*S16*(1/2^2)=S16
[ FAILED   ] vxMultiply.Fuzzy/329/WRAP/TO_ZERO 640x480 S16*S16*(1/2^3)=S16
[ FAILED   ] vxMultiply.Fuzzy/330/WRAP/TO_ZERO 640x480 S16*S16*(1/2^4)=S16
[ FAILED   ] vxMultiply.Fuzzy/331/WRAP/TO_ZERO 640x480 S16*S16*(1/2^5)=S16
[ FAILED   ] vxMultiply.Fuzzy/332/WRAP/TO_ZERO 640x480 S16*S16*(1/2^6)=S16
[ FAILED   ] vxMultiply.Fuzzy/333/WRAP/TO_ZERO 640x480 S16*S16*(1/2^7)=S16
[ FAILED   ] vxMultiply.Fuzzy/334/WRAP/TO_ZERO 640x480 S16*S16*(1/2^8)=S16
[ FAILED   ] vxMultiply.Fuzzy/335/WRAP/TO_ZERO 640x480 S16*S16*(1/2^9)=S16
[ FAILED   ] vxMultiply.Fuzzy/336/WRAP/TO_ZERO 640x480 S16*S16*(1/2^10)=S16
[ FAILED   ] vxMultiply.Fuzzy/337/WRAP/TO_ZERO 640x480 S16*S16*(1/2^11)=S16
[ FAILED   ] vxMultiply.Fuzzy/338/WRAP/TO_ZERO 640x480 S16*S16*(1/2^12)=S16
[ FAILED   ] vxMultiply.Fuzzy/339/WRAP/TO_ZERO 640x480 S16*S16*(1/2^13)=S16
[ FAILED   ] vxMultiply.Fuzzy/340/WRAP/TO_ZERO 640x480 S16*S16*(1/2^14)=S16
[ FAILED   ] vxMultiply.Fuzzy/341/WRAP/TO_ZERO 640x480 S16*S16*(1/2^15)=S16
[ FAILED   ] GaussianPyramid.GraphProcessing/1/randomInput/VX_BORDER_UNDEFINED/sz=16x16/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.GraphProcessing/3/randomInput/VX_BORDER_UNDEFINED/sz=256x256/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.GraphProcessing/5/randomInput/VX_BORDER_UNDEFINED/sz=640x480/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.GraphProcessing/37/lena/VX_BORDER_UNDEFINED/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.ImmediateProcessing/1/randomInput/VX_BORDER_UNDEFINED/sz=16x16/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.ImmediateProcessing/3/randomInput/VX_BORDER_UNDEFINED/sz=256x256/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.ImmediateProcessing/5/randomInput/VX_BORDER_UNDEFINED/sz=640x480/VX_SCALE_PYRAMID_ORB
[ FAILED   ] GaussianPyramid.ImmediateProcessing/37/lena/VX_BORDER_UNDEFINED/VX_SCALE_PYRAMID_ORB
[ DISABLED ] 8190 test(s)