`rebar compile` doesn't create `cl_drv.so` file

[nivertech]

Running rebar compile doesn't compile native code in the same way sa make does.

Running rebar clean compile does create priv/cl_drv.so, but cl app expect it to be under lib/{release,debug}/{32,64}.

I think the more standard way is to have everything under priv.

[cl (zvi_rebar_getdeps_220112)]$ make clean
(cd c_src; make clean)
make[1]: Entering directory `/home/zvi/ws/cl/c_src'
make clean_internal TYPE=release WORDSIZE=32
make[2]: Entering directory `/home/zvi/ws/cl/c_src'
rm -f ../obj/release/32/*.o
rm -f ../lib/release/32/*.so
make[2]: Leaving directory `/home/zvi/ws/cl/c_src'
make clean_internal TYPE=debug   WORDSIZE=32
make[2]: Entering directory `/home/zvi/ws/cl/c_src'
rm -f ../obj/debug/32/*.o
rm -f ../lib/debug/32/*.so
make[2]: Leaving directory `/home/zvi/ws/cl/c_src'
make clean_internal TYPE=release WORDSIZE=64
make[2]: Entering directory `/home/zvi/ws/cl/c_src'
rm -f ../obj/release/64/*.o
rm -f ../lib/release/64/*.so
make[2]: Leaving directory `/home/zvi/ws/cl/c_src'
make clean_internal TYPE=debug   WORDSIZE=64
make[2]: Entering directory `/home/zvi/ws/cl/c_src'
rm -f ../obj/debug/64/*.o
rm -f ../lib/debug/64/*.so
make[2]: Leaving directory `/home/zvi/ws/cl/c_src'
make[1]: Leaving directory `/home/zvi/ws/cl/c_src'
(cd src; make clean)
make[1]: Entering directory `/home/zvi/ws/cl/src'
rm -f ../ebin/cl.beam ../ebin/clu.beam
make[1]: Leaving directory `/home/zvi/ws/cl/src'
[cl (zvi_rebar_getdeps_220112)]$ rebar compile
==> cl (compile)
Compiled src/clu.erl
Compiled src/cl.erl
[cl (zvi_rebar_getdeps_220112)]$ rebar clean compile
==> cl (clean)
==> cl (compile)
Compiled src/clu.erl
Compiled src/cl.erl
Compiling c_src/cl_hash.c
Compiling c_src/cl_nif.c

cl_drv.so under priv created:

[cl (zvi_rebar_getdeps_220112)]$ ls priv/
cl_drv.so

but all the other folders are empty:

[cl (zvi_rebar_getdeps_220112)]$ ls {lib,obj}/*/*
lib/debug/32:

lib/debug/64:

lib/release/32:

lib/release/64:

obj/debug/32:

obj/debug/64:

obj/release/32:

obj/release/64:
[cl (zvi_rebar_getdeps_220112)]$ 

[nivertech]

One way to fix it is to add rebar.config file with post-hook:

%% == Pre/Post Command Hooks ==

{post_hooks, [{clean, "make clean"},
              {compile, "make"}]}.

As in this commit:

https://github.com/nivertech/cl/commit/4dca829c805dbb22c2f2def004b73673650b955c

[tonyrog]

Since I am developing for multiple versions and releases I tend to compile for both 32- and 64- bit (debug/release) at the same to save some time. I guess I am the only one that does it like this? I will try to get some time to make a rebar compatible release soon. I will also push som 1.2 fixes soon.

[nivertech]

For development you may add make target to compile all combinations 4 combinations:

• {C debug, C release} x {Arch: x86, x86_64, ...}

But for the end user, who added cl as rebar dependency to his project, the build should guess the best configuration, like many other rebarified erlang projects with native code (i.e. with c_src folder).

[tonyrog]

I have made a major cleanup. But not yet in master. You will find the result in branch 'dev'. It should compile with and without rebar. I removed a lot of junk so I may have removed to much. I also need to beg Dan to have a look ion the effects on the win32 platform. Please have a look. Thanks. Tony.

[nivertech]

I tested branch dev here:

https://github.com/nivertech/cl_examples/tree/zvi_test_cl_dev

My system is Ubuntu 10.04 32 bit with AMD APP SDK.

rebar clean / rebar compile works.

[cl ((077aaff...))]$ rebar clean
==> cl (clean)
[cl ((077aaff...))]$ cd priv/
[priv ((077aaff...))]$ ls
[priv ((077aaff...))]$ cd ..
[cl ((077aaff...))]$ rebar compile
==> cl (compile)
Compiled src/clu.erl
Compiled src/cl10.erl
Compiled src/cl.erl
Compiled src/cl11.erl
Compiled src/cl13.erl
Compiled src/cl12.erl
Compiling c_src/cl_nif.c
Compiling c_src/cl_hash.c
[cl ((077aaff...))]$ cd priv/
[priv ((077aaff...))]$ ls
cl_nif.so
[priv ((077aaff...))]$ cd ..

Most examples work, except:

1> cl_basic:test().
PlatformInfo: [{profile,"FULL_PROFILE"},
               {version,"OpenCL 1.2 AMD-APP (1113.2)"},
               {name,"AMD Accelerated Parallel Processing"},
               {vendor,"Advanced Micro Devices, Inc."},
               {extensions,"cl_khr_icd cl_amd_event_callback cl_amd_offline_devices"}]
Device: {device_t,145620232,<<>>}
** exception error: bad argument
     in function  cl:get_device_info/2
        called as cl:get_device_info({device_t,145620232,<<>>},preferred_vector_width_half)
     in call from cl:get_info_list/5 (src/cl.erl, line 2172)
     in call from cl_basic:'-test/1-fun-0-'/1 (src/cl_basic.erl, line 20)
     in call from lists:foreach/2 (lists.erl, line 1262)
     in call from cl_basic:test/1 (src/cl_basic.erl, line 17)

and

7> cl_test:test0().
** exception error: no match of right hand side value {error,invalid_platform}
     in function  cl_test:test0/1 (src/cl_test.erl, line 19)
8> cl_test:test1().
** exception error: no match of right hand side value {error,invalid_platform}
     in function  cl_test:test1/2 (src/cl_test.erl, line 56)

[tonyrog]

Nice. Thanks. I guess some filtering is needed in cl_basic / get_device_info when an info item is not supported in the driver.

and

Maybe you do not have a cpu device ?

[nivertech]

No, I have CPU device only. I think the behavior is changed from master, I.e. the output of cl_basic:test() was exactly the same as in #13 .

[tonyrog]

Ok, I found it. A bit too much device_info for openCL 1.0. At least it works on my Linux machines right now.

[tonyrog]

Or what do cl:versions() return on your machine?

[nivertech]

With lastest dev branch:

1> cl:versions().
[{1,0}]


3> cl_test:test0().
** exception error: no match of right hand side value {error,invalid_platform}
     in function  cl_test:test0/1 (src/cl_test.erl, line 19)
4> cl_test:test1().
** exception error: no match of right hand side value {error,invalid_platform}
     in function  cl_test:test1/2 (src/cl_test.erl, line 56)


9> cl_basic:test().
PlatformInfo: [{profile,"FULL_PROFILE"},
               {version,"OpenCL 1.2 AMD-APP (1113.2)"},
               {name,"AMD Accelerated Parallel Processing"},
               {vendor,"Advanced Micro Devices, Inc."},
               {extensions,"cl_khr_icd cl_amd_event_callback cl_amd_offline_devices"}]
Device: {device_t,169058000,<<>>}
DeviceInfo: [{type,[cpu]},
             {vendor_id,4098},
             {max_compute_units,2},
             {max_work_item_dimensions,3},
             {max_work_group_size,1024},
             {max_work_item_sizes,[1024,1024,1024]},
             {preferred_vector_width_char,16},
             {preferred_vector_width_short,8},
             {preferred_vector_width_int,4},
             {preferred_vector_width_long,2},
             {preferred_vector_width_float,4},
             {preferred_vector_width_double,2},
             {max_clock_frequency,800},
             {address_bits,32},
             {max_read_image_args,128},
             {max_write_image_args,8},
             {max_mem_alloc_size,1073741824},
             {image2d_max_width,8192},
             {image2d_max_height,8192},
             {image3d_max_width,2048},
             {image3d_max_height,2048},
             {image3d_max_depth,2048},
             {image_support,true},
             {max_parameter_size,4096},
             {max_samplers,16},
             {mem_base_addr_align,1024},
             {min_data_type_align_size,128},
             {single_fp_config,[denorm,inf_nan,round_to_nearest,round_to_zero,
                                round_to_inf,fma]},
             {global_mem_cache_type,read_write},
             {global_mem_cacheline_size,64},
             {global_mem_cache_size,32768},
             {global_mem_size,3758096384},
             {max_constant_buffer_size,65536},
             {max_constant_args,8},
             {local_mem_type,global},
             {local_mem_size,32768},
             {error_correction_support,false},
             {profiling_timer_resolution,1},
             {endian_little,true},
             {available,true},
             {compiler_available,true},
             {execution_capabilities,[kernel,native_kernel]},
             {queue_properties,[profiling_enable]},
             {name,"Intel(R) Core(TM)2 Duo CPU     P8600  @ 2.40GHz"},
             {vendor,"GenuineIntel"},
             {driver_version,"1113.2 (sse2)"},
             {profile,"FULL_PROFILE"},
             {version,"OpenCL 1.2 AMD-APP (1113.2)"},
             {extensions,"cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_ext_device_fission cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_popcnt "},
             {platform,{platform_t,169057944,<<>>}}]
ContextInfo: [{reference_count,1},
              {devices,[{device_t,169058000,<<>>}]},
              {properties,[]}]
Context2: [{reference_count,1},
           {devices,[{device_t,169058000,<<>>}]},
           {properties,[]}]
Queue: {command_queue_t,3008851184,<<>>}
QueueInfo: [{context,{context_t,3008850328,<<>>}},
            {device,{device_t,169058000,<<>>}},
            {reference_count,1},
            {properties,[]}]
Sampler1: {sampler_t,3008874352,<<>>}
Sampler1Info: [{reference_count,1},
               {context,{context_t,3008850328,<<>>}},
               {normalized_coords,true},
               {addressing_mode,clamp},
               {filter_mode,nearest}]
** exception error: no match of right hand side value {error,invalid_value}
     in function  cl_basic:test_sampler/2 (src/cl_basic.erl, line 148)
     in call from lists:foreach/2 (lists.erl, line 1262)
     in call from cl_basic:test/1 (src/cl_basic.erl, line 34)
10> 

[tonyrog]

strange, the versions says {1,0} only but the platform is {1,2} could you please try to pull the dev branch again an rebuild ? I think that cl_test:test() is trying to find a default platform, maybe that is not supported anymore, I need to check that as well

[nivertech]

I run with latest dev branch and still getting:

3> cl:versions().
[{1,0}]

In order to rule out, that it's problem with my laptop's environment, I'll check it soon on clean VM on AWS EC2.