pti-gpu icon indicating copy to clipboard operation
pti-gpu copied to clipboard

ompt_callback_target_data_op_t strange behavior

Open TApplencourt opened this issue 2 years ago • 3 comments

Not really a bug in PTI (because it produce correct results) maybe more a bug in the implementation in icpx. You seem to be the POC for this kind of bug. Sorry :)

Official headers for OpenMP tools say

typedef void (*ompt_callback_target_data_op_t)(
    ompt_id_t target_id, ompt_id_t host_op_id, ompt_target_data_op_t optype,
    void *src_addr, int src_device_num, void *dest_addr, int dest_device_num,
    size_t bytes, const void *codeptr_ra);

https://github.com/OpenMP/sources/blob/eb82823fda8b3ead31da8a517946d9219b41f440/include/omp-tools.h#L692-L695

Where in PTI you are prepending an ompt_scope_endpoint_t endpoint to the argument list https://github.com/intel/pti-gpu/blob/6e18132fafaff41fff306a0a0421fbadfd01697a/samples/omp_hot_regions/tool.cc#L73-L78

Strangely enough, this "non-OpeMP 5.2 compliant" parameters list seems needed to get correct results with public icpx/omp runtime.

TApplencourt avatar Mar 16 '22 23:03 TApplencourt

Hi @TApplencourt, that's right, our current compiler has not claimed OpenMP 5.2 compatibility yet. It's planned by the end of this year.

anton-v-gorshkov avatar Mar 17 '22 13:03 anton-v-gorshkov

Thanks for the info! And thanks a lot for the example. It saved us some hours of debugging :) I let you decide if you want to close the issue or not. I don't mind either way

TApplencourt avatar Mar 17 '22 14:03 TApplencourt

The issue here is, that OneAPI implements the _emi callbacks as their normal ones. This affects at least target, target_data_op and target_submit. You can still register callbacks with the callback signature from the OpenMP specifications. However, this causes issues as the passed values are shifted.

Using the _emi callbacks causes OneAPI to not call the callbacks at all.

Thyre avatar Feb 03 '23 18:02 Thyre