OMPT: device_num is -1

[ZealotTKO]

I added the following to this repo's OMPT example:

  #pragma omp target data map(tofrom: initial_device)
  {
      int a;
      #pragma omp target
      {
          for(int i=0; i<100; i++){
          a+=i;
        }
      }
  }
  #pragma omp target data map(tofrom: initial_device) device(1)
  {
      int a;
      #pragma omp target
      {
          for(int i=0; i<100; i++){
          a+=i;
        }
      }
  }

The corresponding output I see is:

Callback Target: target_id=1 kind=2 endpoint=1 device_num=-1 code=0x206615
  Callback DataOp: target_id=1 host_op_id=2 optype=1 src=0x7fff47f9b2dc src_device_num=0 dest=(nil) dest_device_num=0 bytes=4 code=0x7fb0354d5500
  Callback DataOp: target_id=1 host_op_id=3 optype=2 src=0x7fff47f9b2dc src_device_num=0 dest=0x7fb028c00000 dest_device_num=0 bytes=4 code=0x7fb0354d53cd
Callback Target: target_id=1 kind=2 endpoint=2 device_num=0 code=0x206615
Callback Target: target_id=4 kind=1 endpoint=1 device_num=0 code=0x206667
Callback Target: target_id=4 kind=1 endpoint=2 device_num=0 code=0x206667
Callback Target: target_id=6 kind=3 endpoint=1 device_num=0 code=0x206699
  Callback DataOp: target_id=6 host_op_id=7 optype=3 src=0x7fb028c00000 src_device_num=0 dest=0x7fff47f9b2dc dest_device_num=0 bytes=4 code=0x7fb0354f20b6
  Callback DataOp: target_id=6 host_op_id=8 optype=4 src=0x7fb028c00000 src_device_num=0 dest=(nil) dest_device_num=0 bytes=0 code=0x7fb0354d6d2b
Callback Target: target_id=6 kind=3 endpoint=2 device_num=0 code=0x206699
Callback Target: target_id=9 kind=2 endpoint=1 device_num=1 code=0x2066dd
  Callback DataOp: target_id=9 host_op_id=10 optype=1 src=0x7fff47f9b2dc src_device_num=1 dest=(nil) dest_device_num=0 bytes=4 code=0x7fb0354d5500
  Callback DataOp: target_id=9 host_op_id=11 optype=2 src=0x7fff47f9b2dc src_device_num=0 dest=0x7fb028600000 dest_device_num=1 bytes=4 code=0x7fb0354d53cd
Callback Target: target_id=9 kind=2 endpoint=2 device_num=1 code=0x2066dd
Callback Target: target_id=12 kind=1 endpoint=1 device_num=0 code=0x20672f
Callback Target: target_id=12 kind=1 endpoint=2 device_num=0 code=0x20672f
Callback Target: target_id=14 kind=3 endpoint=1 device_num=1 code=0x20675f
  Callback DataOp: target_id=14 host_op_id=15 optype=3 src=0x7fb028600000 src_device_num=1 dest=0x7fff47f9b2dc dest_device_num=0 bytes=4 code=0x7fb0354f20b6
  Callback DataOp: target_id=14 host_op_id=16 optype=4 src=0x7fb028600000 src_device_num=1 dest=(nil) dest_device_num=0 bytes=0 code=0x7fb0354d6d2b
Callback Target: target_id=14 kind=3 endpoint=2 device_num=1 code=0x20675f
Callback Fini: device_num=0
Callback Fini: device_num=1

It appears ompt_callback_target_emi's device_num is -1 when using target constructs without the device(x) clause. I would expect it to be OMP_DEFAULT_DEVICE. Note the second construct that add the clause device(1) sets device_num appropriately.

[dhruvachak]

Fixed with this commit.

https://github.com/RadeonOpenCompute/llvm-project/commit/62438c20322011436cb35469364957942c8f8883

[jplehr]

This is apparently broken again, due to some change upstream. Currently working on a fix.

[jplehr]

Should be fixed (again) with https://github.com/RadeonOpenCompute/llvm-project/commit/ee4109c2fb23ebbb4f43b7c6f234337a9390c1d4

[jplehr]

This is fixed and covered by a test.