Successful Linux Installs - How You Did It?

[keith-dont-go]

Hey gang - I just thought it would be cool to have a thread in which the steps / techniques of successful linux installs were recorded / discussed. I've done it now on three different HP machines of various vintages that were originally Windows XP, 10, and 10 respectively, but are now all the latest Linux Mint. I can now do these installs quickly and with confidence. But I didn't get to this point without a lot of floundering around and observing what worked and what didn't. OK I'll leave it at that for now - we'll see how this goes.

[keith-dont-go]

For me the first step is to have a pretty scrupulously maintained packaging system. I use Mint's Synaptic Package Manager. Then I'll install some other program that uses opencv and opencl. For good luck. Then I install the Intel opencl stuff from deb files that I converted from RPM files using something called "alien" IIRC.

Then I install clinfo. The package manager will also install the beignet ICD loader. This is my "weak" machine but it still has opencl and runs much cooler than my core i7 7700 CPU / HD graphics Kaby Lake etc. Here's what clinfo says about this weak machine. If you can get to this point your chances of a successful installation are very good. Stay tuned:

---

Number of platforms 2 Platform Name Intel(R) OpenCL Platform Vendor Intel(R) Corporation Platform Version OpenCL 1.2 Platform Profile FULL_PROFILE Platform Extensions cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_depth_images cl_khr_3d_image_writes cl_intel_exec_by_local_thread cl_khr_spir Platform Extensions function suffix INTEL

Platform Name Intel Gen OCL Driver Platform Vendor Intel Platform Version OpenCL 1.2 beignet 1.1.1 Platform Profile FULL_PROFILE Platform Extensions cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_spir cl_khr_icd Platform Extensions function suffix Intel

Platform Name Intel(R) OpenCL Number of devices 1 Device Name Intel(R) Pentium(R) CPU J2900 @ 2.41GHz Device Vendor Intel(R) Corporation Device Vendor ID 0x8086 Device Version OpenCL 1.2 (Build 475) Driver Version 1.2.0.475 Device OpenCL C Version OpenCL C 1.2 Device Type CPU Device Profile FULL_PROFILE Max compute units 4 Max clock frequency 2410MHz Device Partition (core) Max number of sub-devices 4 Supported partition types by counts, equally, by names (Intel) Max work item dimensions 3 Max work item sizes 8192x8192x8192 Max work group size 8192 Preferred work group size multiple 128 Preferred / native vector sizes
char 1 / 16
short 1 / 8
int 1 / 4
long 1 / 2
half 0 / 0 (n/a) float 1 / 4
double 0 / 0 (n/a) Half-precision Floating-point support (n/a) Single-precision Floating-point support (core) Denormals Yes Infinity and NANs Yes Round to nearest Yes Round to zero No Round to infinity No IEEE754-2008 fused multiply-add No Support is emulated in software No Correctly-rounded divide and sqrt operations No Double-precision Floating-point support (n/a) Address bits 64, Little-Endian Global memory size 4016787456 (3.741GiB) Error Correction support No Max memory allocation 1004196864 (957.7MiB) Unified memory for Host and Device Yes Minimum alignment for any data type 128 bytes Alignment of base address 1024 bits (128 bytes) Global Memory cache type Read/Write Global Memory cache size 1048576 Global Memory cache line 64 bytes Image support Yes Max number of samplers per kernel 480 Max size for 1D images from buffer 62762304 pixels Max 1D or 2D image array size 2048 images Max 2D image size 16384x16384 pixels Max 3D image size 2048x2048x2048 pixels Max number of read image args 480 Max number of write image args 480 Local memory type Global Local memory size 32768 (32KiB) Max constant buffer size 131072 (128KiB) Max number of constant args 480 Max size of kernel argument 3840 (3.75KiB) Queue properties
Out-of-order execution Yes Profiling Yes Local thread execution (Intel) Yes Prefer user sync for interop No Profiling timer resolution 1ns Execution capabilities
Run OpenCL kernels Yes Run native kernels Yes SPIR versions 1.2 printf() buffer size 1048576 (1024KiB) Built-in kernels
Device Available Yes Compiler Available Yes Linker Available Yes Device Extensions cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_depth_images cl_khr_3d_image_writes cl_intel_exec_by_local_thread cl_khr_spir

Platform Name Intel Gen OCL Driver Number of devices 1 Device Name Intel(R) HD Graphics Bay Trail-T Device Vendor Intel Device Vendor ID 0x8086 Device Version OpenCL 1.2 beignet 1.1.1 Driver Version 1.1.1 Device OpenCL C Version OpenCL C 1.2 beignet 1.1.1 Device Type GPU Device Profile FULL_PROFILE Max compute units 4 Max clock frequency 1000MHz Device Partition (core) Max number of sub-devices 1 Supported partition types None, None, None Max work item dimensions 3 Max work item sizes 256x256x256 Max work group size 256 Preferred work group size multiple 16 Preferred / native vector sizes
char 16 / 8
short 8 / 8
int 4 / 4
long 2 / 2
half 0 / 8 (n/a) float 4 / 4
double 0 / 2 (n/a) Half-precision Floating-point support (n/a) Single-precision Floating-point support (core) Denormals No Infinity and NANs Yes Round to nearest Yes Round to zero No Round to infinity No IEEE754-2008 fused multiply-add No Support is emulated in software No Correctly-rounded divide and sqrt operations No Double-precision Floating-point support (n/a) Address bits 32, Little-Endian Global memory size 2147483648 (2GiB) Error Correction support No Max memory allocation 1073741824 (1024MiB) Unified memory for Host and Device Yes Minimum alignment for any data type 128 bytes Alignment of base address 1024 bits (128 bytes) Global Memory cache type Read/Write Global Memory cache size 8192 Global Memory cache line 64 bytes Image support Yes Max number of samplers per kernel 16 Max size for 1D images from buffer 65536 pixels Max 1D or 2D image array size 2048 images Max 2D image size 8192x8192 pixels Max 3D image size 8192x8192x2048 pixels Max number of read image args 128 Max number of write image args 8 Local memory type Global Local memory size 65536 (64KiB) Max constant buffer size 134217728 (128MiB) Max number of constant args 8 Max size of kernel argument 1024 Queue properties
Out-of-order execution No Profiling Yes Prefer user sync for interop Yes Profiling timer resolution 80ns Execution capabilities
Run OpenCL kernels Yes Run native kernels Yes SPIR versions <printDeviceInfo:138: get SPIR versions size : error -30> printf() buffer size 1048576 (1024KiB) Built-in kernels __cl_copy_region_align4;__cl_copy_region_align16;__cl_cpy_region_unalign_same_offset;__cl_copy_region_unalign_dst_offset;__cl_copy_region_unalign_src_offset;__cl_copy_buffer_rect;__cl_copy_image_1d_to_1d;__cl_copy_image_2d_to_2d;__cl_copy_image_3d_to_2d;__cl_copy_image_2d_to_3d;__cl_copy_image_3d_to_3d;__cl_copy_image_2d_to_buffer;__cl_copy_image_3d_to_buffer;__cl_copy_buffer_to_image_2d;__cl_copy_buffer_to_image_3d;__cl_fill_region_unalign;__cl_fill_region_align2;__cl_fill_region_align4;__cl_fill_region_align8_2;__cl_fill_region_align8_4;__cl_fill_region_align8_8;__cl_fill_region_align8_16;__cl_fill_region_align128;__cl_fill_image_1d;__cl_fill_image_1d_array;__cl_fill_image_2d;__cl_fill_image_2d_array;__cl_fill_image_3d; Device Available Yes Compiler Available Yes Linker Available Yes Device Extensions cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_spir cl_khr_icd

NULL platform behavior clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) No platform clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) No platform clCreateContext(NULL, ...) [default] No platform clCreateContext(NULL, ...) [other] Success [INTEL] clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) No platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) No platform

---

[keith-dont-go]

Shifting gears a little bit. The installation on my core i7 7700 was working fine but would seg fault upon completion. The Pentium J2900 / HD Graphics Bay Trail-T described previously does not seg fault and is now running in massive nested loops to test the range of options like --winsize, --levels, --poly-n. But it's pretty weak and not the fastest thing in town. So I wanted to fix the seg faulting on my big iron.

The first thing I did was blow the entire installation directory away. That's the beauty of installing in development mode. Then I made this script from the instructions:

---

#!/bin/bash sudo apt-get install git virtualenv python-dev ocl-icd-opencl-dev libopencv-dev python-opencv ffmpeg git clone https://github.com/dthpham/butterflow.git virtualenv -p /usr/bin/python2 butterflow cd ./butterflow source ./bin/activate echo "/usr/lib/python2.7/dist-packages" > lib/python2.7/site-packages/butterflow.pth python setup.py develop

---

It will ask you for your sudo password. It jumps into the development mode during the install then exits when the script exits. No problemo. So now to actually run the program you have to jump into the install directory and run

source ./bin/activate

And that gives you a butterflow prompt and puts you into the development mode / environment in which all your paths and environment variables will be kosher. I think that's the idea. Then you can type:

butterflow -d

And you might see something like this:

OpenCL devices: Platform : Intel(R) OpenCL Platform Vendor : Intel(R) Corporation Platform Version : OpenCL 2.0

• Device 0 : Intel(R) HD Graphics Version : OpenCL 2.0 Version : r5.0.63503 Work Sizes : 256, 256x256x256 Compatible : Yes
• Device 1 : Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz Version : OpenCL 2.0 Version : 1.2.0.475 Work Sizes : 8192, 8192x8192x8192 Compatible : Yes

Nice. But does it run and did the reinstall fix the seg fault? Yes, and yes:

(butterflow) >butterflow -s a=0,b=end,spd=0.187 -v carnera.mp4 -o testoutpath.mp4 [butterflow:INFO]: Version 0.2.3 [butterflow:INFO]: Cache directory: /tmp/butterflow-0.2.3 [butterflow:INFO]: At least one compatible OpenCL device was detected [butterflow:INFO]: Using device: Intel(R) HD Graphics (autoselected) [butterflow:INFO]: Hardware acceleration is enabled [butterflow:INFO]: Rendering: [butterflow:INFO]: Sequence: Duration=0:00:32.241000 (32.24s), Frames=807, Rate=25.0302410126 (-----etcetera-----) [butterflow:INFO]: Frames in region: 0-806 [butterflow:INFO]: Region length: 807 [butterflow:INFO]: Region duration: 32.241000s [butterflow:INFO]: Number of frame pairs: 806 [butterflow:INFO]: Interpolation rate: 5 [butterflow:INFO]: Time stepping: 0.167,0.333,0.500,0.667,0.833 [butterflow:INFO]: Frames to write: 4315 [butterflow:INFO]: Will interpolate: 4836 [butterflow:INFO]: Extra frames (to discard): 521 [butterflow:INFO]: Drop every: 9 [butterflow:INFO]: Dupe every: 0 [butterflow:INFO]: Ready to run: 807 times [butterflow:INFO]: Showing a sample of the first and last 15 runs: [butterflow:INFO]: To write: S0 I4,1 0.12% [butterflow:INFO]: To write: S1 I4,1 0.23% [butterflow:INFO]: To write: S2 I4,1 0.35% (-----etcetera-----) [butterflow:INFO]: <Rendering progress: 9.39%> [butterflow:INFO]: <Rendering progress: 18.82%> (-----etcetera-----) [butterflow:INFO]: Run 806 (this is the final run): [butterflow:INFO]: To write: S806 (-----etcetera-----) [butterflow:WARNING]: Duping S806 [butterflow:INFO]: Done rendering Subregion (0) [butterflow:INFO]: [Subprocess] Closing pipe to the video writer [butterflow:INFO]: Rendering is finished [butterflow:INFO]: Moving: carnera.8791.mp4 -> testoutpath.mp4 [butterflow:INFO]: Write ratio: 4315/4315, (100.00%) [butterflow:INFO]: Final output frames: 807 source, +3443 interpolated, +65 duped, -0 dropped [butterflow:INFO]: Output file size: 28713.61 kB (23836.79 kB) [butterflow:INFO]: Rendering took 2.58 mins, done. (butterflow) >

Clean exit - gave me back the (butterflow) prompt. No bad news. Pretty heavy interpolation load. So now that I'm not seg faulting I'm gonna put this machine into big loops to test out all the options and get my answers quicker. The other machine is still soldiering along after several hours but this one will overtake it soon and complete the job quicker. The frame size in the video just ran is 640x480, 3443 new frames were interpolated, in just 2 minutes 35 seconds. Not too shabby IMHO. Note: the Intel opencl stuff was also installed as described in the previous post and this machine did NOT like the beignet_opencl_icd so I had to remove it. Anyhoo YMMV.

[dthpham]

Hey thanks for documenting this. I've been going through all my machines/platforms and reinstalling again and will update the docs with what you have and what I come up with so that others have an easier time building and running this thing.

[keith-dont-go]

You bet. I'm working on something similar to the python optical flow demos. And I got the same error I got a few weeks ago with butterflow: "OpenCV Error: Gpu API call (-1001) in initializeOpenCLDevices, file blah blah blah cl_context.cpp blah blah blah OpenCV Error: Unknown error code -221 (OpenCL not available) in getContext".

So for some reason I'd "lost" my intel opencl capability. The intel stuff is the ONLY stuff I could get to work for both CPU and GPU. So I had to reinstall it. Here are the steps in detail. Remember this is INTEL hardware - and I'm not anything even close to an expert. Proceed at your own risk.

1. Go here https://software.intel.com/en-us/articles/opencl-drivers#latest_linux_driver
2. Download this "intel-opencl-r5.0 (SRB5.0) Linux driver package" (filename: SRB5.0_linux64.zip).
3. Double click (I'm using linux Mint) on the zip file and you'll see: intel-opencl-r5.0-63503.x86_64.rpm, intel-opencl-devel-r5.0-63503.x86_64.rpm, and intel-opencl-cpu-r5.0-63503.x86_64.rpm.
4. Drag them out and use "alien" (/usr/bin/alien) to convert these into deb files. Here's what I followed IIRC: https://software.intel.com/en-us/forums/opencl/topic/390630
5. Now you have these files: intel-opencl_0r5.0-63504_amd64.deb, intel-opencl-cpu_0r5.0-63504_amd64.deb, and intel-opencl-devel_0r5.0-63504_amd64.deb.
6. At this point I just clicked on the deb files and they kinda installed themselves.

Good luck YMMV. Here's the abbreviated result from clinfo:

` Platform Name Intel(R) OpenCL Number of devices 2 Device Name Intel(R) HD Graphics Device Vendor Intel(R) Corporation Device Vendor ID 0x8086 Device Version OpenCL 2.0 Driver Version r5.0.63503 Device OpenCL C Version OpenCL C 2.0 Device Type GPU Device Profile FULL_PROFILE Max compute units 23 Max clock frequency 1150MHz

Device Name Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz Device Vendor Intel(R) Corporation Device Vendor ID 0x8086 Device Version OpenCL 2.0 (Build 475) Driver Version 1.2.0.475 Device OpenCL C Version OpenCL C 2.0 Device Type CPU Device Profile FULL_PROFILE Max compute units 8 Max clock frequency 3600MHz`

[Ravenstine]

Someone made a working Dockerfile for Butterflow, if you know how to use Docker: https://github.com/Saren-Arterius/my-dockerfiles/blob/master/butterflow/Dockerfile

The prebuilt image works for me on macOS: https://hub.docker.com/r/saren/butterflow/

I imagine it works in native Linux, and probably much better.

[ascendant512]

Have written a placeholder ebuild to butterflow-0.2.4_alpha4.ebuild that any other users may be able to adapt to their situation. Mine is fairly hardware specific (i5-4690K) taking into account https://wiki.gentoo.org/wiki/OpenCL butterflow requires python 2.7 and opencv 2 and won't build against opencv 3, however opencv 2 doesn't build against ffmpeg 4.1 (as of this writing) without this patch, which may be added to portage as a userpatch: https://github.com/ilovezfs/opencv/commit/99091a62463ecd4a8e26b17b10f259ea6f104c62.patch butterflow-0.2.4_alpha4.ebuild.txt I also could not get a successful build without implementing the fixes detailed in #101 and #116 and finally the virtualenv command in the install instructions requires --system-site-packages in order for setup.py to work (otherwise you will need to manually copy a ton of the OS's python libraries into the virtualenv).