Altering default device not working with reduction tests

[vsilaev]

It looks like https://github.com/beehive-lab/TornadoVM/pull/61 PR conflicts with recent logic for calculating / checking GridInfo by Thanos Stratikopoulos.

I've tested code in PR on multiple devices and it worked ok, however, after merging develop branch with recent changes by Thanos, it stops working if default device is changed via tornado.properties.

Try to run any reduction tests with tornado.driver=0 and tornado.device=1 (i.e. non default, assuming you have at least 2 OpenCL devices) -- all such tests fail now.

[stratika]

Hi @vsilaev, can you be more specific in order to reproduce it. Are you building with both backends? Or only OpenCL?

Also, can you verify that in the last commit before the merge with id: 5c9e2e714541bfe5a706d06a8b9f09d9ed1c531c the described configuration still works?

[vsilaev]

Yes, I were testing with OpenCL only. You need at least 2 OpenCL devices. After compilation try to run tests with default device - everything should be ok. Then in sdk/etc/tornado.properties set tornado.device=1 (i.e. non-default). Now tests with reduction fail.

[vsilaev]

I've added system-out to corresponding methods of TornadoVM class (in runtime) and see the following sequence... Command:

tornado -Dtornado.unittests.verbose=True -Xmx6g -Dtornado.recover.bailout=True --printBytecodes -Dtornado.driver=0 -Dtornado.device=1 uk.ac.manchester.tornado.unittests.tools.TornadoTestRunner  uk.ac.manchester.tornado.unittests.reductions.TestReductionsLong#testMaxReduction

Output:

STREAM-IN ON Intel(R) OpenCL -- Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
STREAM-IN ON Intel(R) OpenCL -- Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
COMPILE ON Intel(R) OpenCL -- Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
LAUNCH ON Intel(R) OpenCL -- Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
COMPILE ON AMD Accelerated Parallel Processing -- Pitcairn
LAUNCH ON AMD Accelerated Parallel Processing -- Pitcairn

AMD is listed as a default device on my desktop (0:0) and Intel (0:1) is set as a default via command line.

Latest 2 lines corresponds to a reduction task that is automatically created.

[jjfumero]

This works for me:

My Intel HD Graphics is device 0:2

make test-hdgraphics
tornado-test.py -V -J"-Ds0.t0.device=0:2 -Ds0.t1.device=0:2" uk.ac.manchester.tornado.unittests.reductions.TestReductionsFloats

Test: class uk.ac.manchester.tornado.unittests.reductions.TestReductionsFloats
	Running test: testSumFloats              ................  [PASS] 
	Running test: testSumFloatsConstant      ................  [PASS] 
	Running test: testSumFloatsLarge         ................  [PASS] 
	Running test: testSumFloats2             ................  [PASS] 
	Running test: testSumFloats3             ................  [PASS] 
	Running test: testComputeAverage         ................  [PASS] 
	Running test: testMultFloats             ................  [PASS] 
	Running test: testComputePi              ................  [PASS] 
	Running test: testMaxReduction           ................  [PASS] 
	Running test: testMaxReduction2          ................  [PASS] 
	Running test: testMinReduction           ................  [PASS] 
	Running test: testMinReduction2          ................  [PASS] 
Test ran: 12, Failed: 0

These are the bytecodes:

vm: STREAM_IN [0xf09733f] [D@f09733f on Intel(R) OpenCL HD Graphics -- Intel(R) Graphics [0x591b], size=0, offset=0 [event list=-1]
vm: STREAM_IN [0x1ffaf86] [D@1ffaf86 on Intel(R) OpenCL HD Graphics -- Intel(R) Graphics [0x591b], size=0, offset=0 [event list=-1]
vm: LAUNCH task XXX__GENERATED_REDUCE14.t0 - testDouble on Intel(R) OpenCL HD Graphics -- Intel(R) Graphics [0x591b], size=0, offset=0 [event list=0]
vm: LAUNCH task XXX__GENERATED_REDUCE14.reduce_seq14 - rAdd on Intel(R) OpenCL HD Graphics -- Intel(R) Graphics [0x591b], size=0, offset=0 [event list=1]
vm: STREAM_OUT_BLOCKING [0x1ffaf86] [D@1ffaf86 on Intel(R) OpenCL HD Graphics -- Intel(R) Graphics [0x591b], size=0, offset=0 [event list=2]

I think we should deprecate the properties file.

BTW, to print the TornadoVM bytecodes you can use:

tornado --printBytecodes .... 

[vsilaev]

Juan, this is because you are using "per-task" mechanism to select driver (-J"-Ds0.t0.device=0:2 -Ds0.t1.device=0:2"). If you will try to use my proposed "per-vm" mechanism than you'll see the issue: -J"-Dtornado.driver=0 -Dtornado.device=2"

Please review the PR https://github.com/beehive-lab/TornadoVM/pull/65 - it fixes the issue.

[vsilaev]

I think we should deprecate the properties file.

I would rather keep it and collect here all properties used with default values and comments what they are for.

[vsilaev]

BTW, to print the TornadoVM bytecodes you can use:

tornado --printBytecodes .... 

I'm aware of this option, but TornadoVM first collects the whole text and then prints it. When there is an error with "launch" app exits without printing bytecode.