Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error creating context on Julia v0.6 #137

Closed
juliohm opened this issue Jun 25, 2017 · 50 comments
Closed

Error creating context on Julia v0.6 #137

juliohm opened this issue Jun 25, 2017 · 50 comments

Comments

@juliohm
Copy link
Member

juliohm commented Jun 25, 2017

Could you please help in solving this issue?

julia> using OpenCL
julia> cl.create_some_context()
ERROR: ReadOnlyMemoryError()
Stacktrace:
 [1] clGetPlatformIDs(::Int64, ::Ptr{Void}, ::Base.RefValue{UInt32}) at /home/juliohm/.julia/v0.6/OpenCL/src/api.jl:17
 [2] macro expansion at /home/juliohm/.julia/v0.6/OpenCL/src/macros.jl:4 [inlined]
 [3] platforms() at /home/juliohm/.julia/v0.6/OpenCL/src/platform.jl:21
 [4] create_some_context() at /home/juliohm/.julia/v0.6/OpenCL/src/context.jl:235
@juliohm
Copy link
Member Author

juliohm commented Jul 2, 2017

Is this an error that you can reproduce?

@dfdx
Copy link
Contributor

dfdx commented Jul 2, 2017

Can't reproduce using latest master:

julia> using OpenCL

julia> cl.create_some_context()
OpenCL.Context(@0x0000000003c89d00 on GeForce GTX 960M)

@juliohm
Copy link
Member Author

juliohm commented Jul 2, 2017

@dfdx, I am using the Beignet driver on an Intel integrated graphics card:

julia> versioninfo()
Julia Version 0.6.1-pre.0
Commit dcf39a1dda* (2017-06-19 13:06 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, skylake)

Beignet version:

Name : beignet
Version : 1.3.1-3
Description : An open source OpenCL implementation for Intel IvyBridge+
iGPUs
Architecture : x86_64
URL : https://01.org/beignet
Licenses : LGPL
Groups : None
Provides : opencl-intel opencl-driver
Depends On : glu llvm mesa opencl-headers
Optional Deps : None
Required By : None
Optional For : ocl-icd
Conflicts With : opencl-intel
Replaces : None
Installed Size : 37.39 MiB
Packager : Laurent Carlier [email protected]
Build Date : Fri 19 May 2017 01:44:25 AM PDT
Install Date : Sat 24 Jun 2017 07:04:08 PM PDT
Install Reason : Explicitly installed
Install Script : No
Validated By : Signature

Graphics card:

Number of platforms 1
Platform Name Intel Gen OCL Driver
Platform Vendor Intel
Platform Version OpenCL 2.0 beignet 1.3
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_3d_image_writes cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_icd cl_intel_accelerator cl_intel_subgroups cl_intel_subgroups_short cl_khr_gl_sharing
Platform Extensions function suffix Intel

Platform Name Intel Gen OCL Driver
Number of devices 1
Device Name Intel(R) HD Graphics Skylake ULT GT2
Device Vendor Intel
Device Vendor ID 0x8086
Device Version OpenCL 2.0 beignet 1.3
Driver Version 1.3
Device OpenCL C Version OpenCL C 2.0 beignet 1.3
Device Type GPU
Device Available Yes
Device Profile FULL_PROFILE
Max compute units 24
Max clock frequency 1000MHz
Device Partition (core)
Max number of sub-devices 1
Supported partition types None, None, None
Max work item dimensions 3
Max work item sizes 512x512x512
Max work group size 512
Compiler Available Yes
Linker Available Yes
Preferred work group size multiple 16
Preferred / native vector sizes
char 16 / 8
short 8 / 8
int 4 / 4
long 2 / 2
half 0 / 8 (cl_khr_fp16)
float 4 / 4
double 0 / 2 (n/a)
Half-precision Floating-point support (cl_khr_fp16)
Denormals No
Infinity and NANs Yes
Round to nearest Yes
Round to zero No
Round to infinity No
IEEE754-2008 fused multiply-add No
Support is emulated in software No
Correctly-rounded divide and sqrt operations No
Single-precision Floating-point support (core)
Denormals No
Infinity and NANs Yes
Round to nearest Yes
Round to zero No
Round to infinity No
IEEE754-2008 fused multiply-add No
Support is emulated in software No
Correctly-rounded divide and sqrt operations No
Double-precision Floating-point support (n/a)
Address bits 32, Little-Endian
Global memory size 4114612224 (3.832GiB)
Error Correction support No
Max memory allocation 3085959168 (2.874GiB)
Unified memory for Host and Device Yes
Shared Virtual Memory (SVM) capabilities (core)
Coarse-grained buffer sharing Yes
Fine-grained buffer sharing No
Fine-grained system sharing No
Atomics No
Minimum alignment for any data type 128 bytes
Alignment of base address 1024 bits (128 bytes)
Preferred alignment for atomics
SVM 0 bytes
Global 0 bytes
Local 0 bytes
Max size for global variable 65536 (64KiB)
Preferred total size of global vars 65536 (64KiB)
Global Memory cache type Read/Write
Global Memory cache size 8192 (8KiB)
Global Memory cache line 64 bytes
Image support Yes
Max number of samplers per kernel 16
Max size for 1D images from buffer 65536 pixels
Max 1D or 2D image array size 2048 images
Base address alignment for 2D image buffers 4096 bytes
Pitch alignment for 2D image buffers 1 bytes
Max 2D image size 8192x8192 pixels
Max 3D image size 8192x8192x2048 pixels
Max number of read image args 128
Max number of write image args 8
Max number of read/write image args 8
Max number of pipe args 16
Max active pipe reservations 1
Max pipe packet size 1024
Local memory type Local
Local memory size 65536 (64KiB)
Max constant buffer size 134217728 (128MiB)
Max number of constant args 8
Max size of kernel argument 1024
Queue properties (on host)
Out-of-order execution No
Profiling Yes
Queue properties (on device)
Out-of-order execution Yes
Profiling Yes
Preferred size 16384 (16KiB)
Max size 262144 (256KiB)
Max queues on device 1
Max events on device 1024
Prefer user sync for interop Yes
Profiling timer resolution 80ns
Execution capabilities
Run OpenCL kernels Yes
Run native kernels Yes
printf() buffer size 1048576 (1024KiB)
Built-in kernels __cl_copy_region_align4;__cl_copy_region_align16;__cl_cpy_region_unalign_same_offset;__cl_copy_region_unalign_dst_offset;__cl_copy_region_unalign_src_offset;__cl_copy_buffer_rect;__cl_copy_image_1d_to_1d;__cl_copy_image_2d_to_2d;__cl_copy_image_3d_to_2d;__cl_copy_image_2d_to_3d;__cl_copy_image_3d_to_3d;__cl_copy_image_2d_to_buffer;__cl_copy_image_3d_to_buffer;__cl_copy_buffer_to_image_2d;__cl_copy_buffer_to_image_3d;__cl_fill_region_unalign;__cl_fill_region_align2;__cl_fill_region_align4;__cl_fill_region_align8_2;__cl_fill_region_align8_4;__cl_fill_region_align8_8;__cl_fill_region_align8_16;__cl_fill_region_align128;__cl_fill_image_1d;__cl_fill_image_1d_array;__cl_fill_image_2d;__cl_fill_image_2d_array;__cl_fill_image_3d;
Device Extensions cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_3d_image_writes cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_icd cl_intel_accelerator cl_intel_subgroups cl_intel_subgroups_short cl_khr_gl_sharing cl_khr_fp16

NULL platform behavior
clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) Intel Gen OCL Driver
clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [Intel]
clCreateContext(NULL, ...) [default] Success [Intel]
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) Success (1)
Platform Name Intel Gen OCL Driver
Device Name Intel(R) HD Graphics Skylake ULT GT2
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1)
Platform Name Intel Gen OCL Driver
Device Name Intel(R) HD Graphics Skylake ULT GT2

ICD loader properties
ICD loader Name OpenCL ICD Loader
ICD loader Vendor OCL Icd free software
ICD loader Version 2.2.11
ICD loader Profile OpenCL 2.1

Do you spot any problem with this configuration right away? I think it was working the first time I compiled Julia v0.6, but I uninstalled beignet from git and installed the stable version from Arch repos. Will anything change if I try to rebuild Julia?

@dfdx
Copy link
Contributor

dfdx commented Jul 2, 2017

I'm not sure about stability of Julia 0.6.1, so I'd start with checking out 0.6.0. No package recompilation should be needed, but just in case you can also run Pkg.build("OpenCL").

The next thing I'd try is to run the same code from pyopencl or any other language. This way we will see whether the error is caused by Julia or driver side.

@juliohm
Copy link
Member Author

juliohm commented Jul 2, 2017

I am using the branch release-0.6, so I think this is already the current stable. I did a pull just now to see if anything changed since the last build.

I will definitely try pyopencl, that is a wonderful idea! If it works there, we will be sure the issue is Julia-related.

@juliohm
Copy link
Member Author

juliohm commented Jul 2, 2017

I can create context with pyopencl just fine, so the issue is indeed Julia-related. What else could I try to debug it further? I don't know where to look next.

@juliohm
Copy link
Member Author

juliohm commented Jul 2, 2017

@dfdx, this is probably the cause:

jump-dev/Ipopt.jl#77

I appreciate if you can take a look at it, OpenCL.jl is quite important for my simulations.

@dfdx
Copy link
Contributor

dfdx commented Jul 2, 2017

I'm not very deep into development of OpenCL.jl itself, so other contributors may be more helpful. But I have a couple of ideas you can try:

  1. Latest master of release-0.6 branch is marked as 0.6.1-pre.0 which doesn't really mean "stable". You may want to try tag v0.6 or, equivalently, just download official binaries for Linux (which should be faster than re-compile everything).
  2. Make sure you are on the latest master of OpenCL.jl itself.
  3. Try running Pkg.test("OpenCL") - this may uncover other errors.
  4. Try instantiating context via cl.create_compute_context(). You may also want to run cl.platforms() since this is what causes the problem from the stacktrace.
  5. Finally, try another driver. I've seen in clBLAS that different drivers exposed different errors, even though Python version worked fine with all drivers.

As far as I can see, our calls to clGetPlatformIDs conforms to specification, so my best bet is that it's either driver implementation or how Julia itself passes data to it.

@juliohm
Copy link
Member Author

juliohm commented Jul 2, 2017

Thank you, I tried with the v0.6 binary and the error is the same. I am on OpenCL master already. The OpenCL tests fail for the same reason. cl.create_compute_context() triggers the same error, as well as cl.platforms(), which is being called under the hood. We can track the error down to that ccall.

I can try another driver, but Beignet was working perfectly fine in previous Julia releases. :/

@tkelman do you think this issue is also due to compiler DLL's? I saw that you fixed similar issue here:

jump-dev/Ipopt.jl#77

@tkelman
Copy link
Contributor

tkelman commented Jul 2, 2017

that issue is windows specific so has nothing to do with this

@dfdx
Copy link
Contributor

dfdx commented Jul 3, 2017

but I uninstalled beignet from git and installed the stable version from Arch repos.

I think it's worth to try to recover the old behavior first, i.e. compile beignet from git again.

@juliohm
Copy link
Member Author

juliohm commented Jul 19, 2017

@vchuravy could you please give a hand on this issue? I tried creating a context with pyopencl and it is working just fine with the Beignet driver:

python> import pyopencl as cl
python> cl.create_some_context(0)
<pyopencl.Context at 0x30787d0 on <pyopencl.Device 'Intel(R) HD Graphics Skylake ULT GT2' on 'Intel Gen OCL Driver' at 0x7f1023dcef60>>

The same attempt with OpenCL.jl triggers an error:

julia> using OpenCL
julia> cl.create_some_context()
ERROR: ReadOnlyMemoryError()
Stacktrace:
 [1] clGetPlatformIDs(::Int64, ::Ptr{Void}, ::Base.RefValue{UInt32}) at /home/juliohm/.julia/v0.6/OpenCL/src/api.jl:17
 [2] macro expansion at /home/juliohm/.julia/v0.6/OpenCL/src/macros.jl:4 [inlined]
 [3] platforms() at /home/juliohm/.julia/v0.6/OpenCL/src/platform.jl:21
 [4] create_some_context() at /home/juliohm/.julia/v0.6/OpenCL/src/context.jl:235

Given this experiment, I am sure this issue is not on the driver side. I appreciate if you can take a look at it when you have some time.

@juliohm
Copy link
Member Author

juliohm commented Oct 2, 2017

Problem solved out of nowhere, probably some driver update.

@juliohm juliohm closed this as completed Oct 2, 2017
@juliohm juliohm reopened this Jan 8, 2018
@juliohm
Copy link
Member Author

juliohm commented Jan 8, 2018

The issue is back, PyOpenCL works fine.

@juliohm
Copy link
Member Author

juliohm commented Jan 8, 2018

I can reproduce the issue with:

nplatforms = Ref{UInt32}()
cl.api.clGetPlatformIDs(0, C_NULL, nplatforms)

ERROR: ReadOnlyMemoryError()
Stacktrace:
 [1] clGetPlatformIDs(::Int64, ::Ptr{Void}, ::Base.RefValue{UInt32}) at /home/juliohm/.julia/v0.6/OpenCL/src/api.jl:15

My clinfo output shows that the clGetPlatormIDs is correctly finding the platforms, and pyopencl can also find it. What can be the cause for the failure with OpenCL.jl?

@iblislin
Copy link

iblislin commented Jan 9, 2018

reduce to ccall?

julia> n = Ref{UInt32}()                                                                                                                          
Base.RefValue{UInt32}(0x0000000b)                                                                                                                 
                                                                                                                                                  
julia> ccall((:clGetPlatformIDs, :libOpenCL), Cint, (Cuint, Ptr{Cuint}, Ptr{UInt32}), 0, C_NULL, n)                                               
0                                                                                                                                                 
                                                                                                                                                  
julia> n                                                                                                                                          
Base.RefValue{UInt32}(0x00000002)

@juliohm
Copy link
Member Author

juliohm commented Jan 9, 2018

Same error @iblis17 , the ccall line throws ReadOnlyMemoryError().

Any suggestion for next debugging steps?

@vchuravy
Copy link
Member

vchuravy commented Jan 12, 2018

You are on Linux right?
What happens if you do:

LD_DEBUG=all julia # Lots of output
julia> ccall((:clGetPlatformIDs, :libOpenCL), Cint, (Cuint, Ptr{Cuint}, Ptr{UInt32}), 0, C_NULL, Ref{UInt32}())

On my system I eventually get something like:

     31148:	calling init: /usr/lib/libOpenCL.so
     31148:	
     31148:	opening file=/usr/lib/libOpenCL.so [0]; direct_opencount=1
     31148:	
     31148:	symbol=clGetPlatformIDs;  lookup in file=/usr/lib/libOpenCL.so [0]
     31148:	binding file /usr/lib/libOpenCL.so [0] to /usr/lib/libOpenCL.so [0]: normal symbol `clGetPlatformIDs'

Telling me that clGetPlatformsIDs has been resolved to /usr/lib/libOpenCL.so. It would also be interesting to see what libOpenCL.so python is resolving to.

My suspicion is that they will be different libOpenCL.so one of them being broken the other one functional.

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

Hi @vchuravy , the first time I run, there is a lot of output, maybe part of it is cropped at the very beginning, but there is no reference to libOpenCL. Is there any way to redirect this output to a file, my terminal is probably not helping.

The second time I run, the output is small, has a reference to libOpenCL in it:


julia> ccall((:clGetPlatformIDs, :libOpenCL), Cint, (Cuint, Ptr{Cuint}, Ptr{UInt32}), 0, C_NULL, Ref{UInt32}())
      3976:	symbol=clGetPlatformIDs;  lookup in file=/usr/lib/libOpenCL.so [0]
      3976:	binding file /usr/lib/libOpenCL.so [0] to /usr/lib/libOpenCL.so [0]: normal symbol `clGetPlatformIDs'
      3976:	symbol=jl_array_grow_end;  lookup in file=julia [0]
      3976:	symbol=jl_array_grow_end;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	binding file julia [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]: normal symbol `jl_array_grow_end'
      3976:	symbol=jl_get_current_task;  lookup in file=julia [0]
      3976:	symbol=jl_get_current_task;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	binding file julia [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]: normal symbol `jl_get_current_task'
      3976:	symbol=jl_array_grow_end;  lookup in file=julia [0]
      3976:	symbol=jl_array_grow_end;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	binding file julia [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]: normal symbol `jl_array_grow_end'
      3976:	symbol=jl_rethrow_other;  lookup in file=julia [0]
      3976:	symbol=jl_rethrow_other;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	binding file julia [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]: normal symbol `jl_rethrow_other'
      3976:	symbol=_ZN4llvm23SimplifyInsertValueInstEPNS_5ValueES1_NS_8ArrayRefIjEERKNS_10DataLayoutEPKNS_17TargetLibraryInfoEPKNS_13DominatorTreeEPNS_15AssumptionCacheEPKNS_11InstructionE;  lookup in file=julia [0]
      3976:	symbol=_ZN4llvm23SimplifyInsertValueInstEPNS_5ValueES1_NS_8ArrayRefIjEERKNS_10DataLayoutEPKNS_17TargetLibraryInfoEPKNS_13DominatorTreeEPNS_15AssumptionCacheEPKNS_11InstructionE;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	symbol=_ZN4llvm23SimplifyInsertValueInstEPNS_5ValueES1_NS_8ArrayRefIjEERKNS_10DataLayoutEPKNS_17TargetLibraryInfoEPKNS_13DominatorTreeEPNS_15AssumptionCacheEPKNS_11InstructionE;  lookup in file=/usr/lib/libdl.so.2 [0]
      3976:	symbol=_ZN4llvm23SimplifyInsertValueInstEPNS_5ValueES1_NS_8ArrayRefIjEERKNS_10DataLayoutEPKNS_17TargetLibraryInfoEPKNS_13DominatorTreeEPNS_15AssumptionCacheEPKNS_11InstructionE;  lookup in file=/usr/lib/librt.so.1 [0]
      3976:	symbol=_ZN4llvm23SimplifyInsertValueInstEPNS_5ValueES1_NS_8ArrayRefIjEERKNS_10DataLayoutEPKNS_17TargetLibraryInfoEPKNS_13DominatorTreeEPNS_15AssumptionCacheEPKNS_11InstructionE;  lookup in file=/usr/lib/libpthread.so.0 [0]
      3976:	symbol=_ZN4llvm23SimplifyInsertValueInstEPNS_5ValueES1_NS_8ArrayRefIjEERKNS_10DataLayoutEPKNS_17TargetLibraryInfoEPKNS_13DominatorTreeEPNS_15AssumptionCacheEPKNS_11InstructionE;  lookup in file=/usr/lib/libc.so.6 [0]
      3976:	symbol=_ZN4llvm23SimplifyInsertValueInstEPNS_5ValueES1_NS_8ArrayRefIjEERKNS_10DataLayoutEPKNS_17TargetLibraryInfoEPKNS_13DominatorTreeEPNS_15AssumptionCacheEPKNS_11InstructionE;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]
      3976:	binding file /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]: normal symbol `_ZN4llvm23SimplifyInsertValueInstEPNS_5ValueES1_NS_8ArrayRefIjEERKNS_10DataLayoutEPKNS_17TargetLibraryInfoEPKNS_13DominatorTreeEPNS_15AssumptionCacheEPKNS_11InstructionE'
      3976:	symbol=_ZN4llvm12hash_combineIJjPNS_5ValueES2_NS_9hash_codeEEEES3_DpRKT_;  lookup in file=julia [0]
      3976:	symbol=_ZN4llvm12hash_combineIJjPNS_5ValueES2_NS_9hash_codeEEEES3_DpRKT_;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	symbol=_ZN4llvm12hash_combineIJjPNS_5ValueES2_NS_9hash_codeEEEES3_DpRKT_;  lookup in file=/usr/lib/libdl.so.2 [0]
      3976:	symbol=_ZN4llvm12hash_combineIJjPNS_5ValueES2_NS_9hash_codeEEEES3_DpRKT_;  lookup in file=/usr/lib/librt.so.1 [0]
      3976:	symbol=_ZN4llvm12hash_combineIJjPNS_5ValueES2_NS_9hash_codeEEEES3_DpRKT_;  lookup in file=/usr/lib/libpthread.so.0 [0]
      3976:	symbol=_ZN4llvm12hash_combineIJjPNS_5ValueES2_NS_9hash_codeEEEES3_DpRKT_;  lookup in file=/usr/lib/libc.so.6 [0]
      3976:	symbol=_ZN4llvm12hash_combineIJjPNS_5ValueES2_NS_9hash_codeEEEES3_DpRKT_;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]
      3976:	binding file /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]: normal symbol `_ZN4llvm12hash_combineIJjPNS_5ValueES2_NS_9hash_codeEEEES3_DpRKT_'
      3976:	symbol=_ZN4llvm19SelectionDAGBuilder16visitInsertValueERKNS_15InsertValueInstE;  lookup in file=julia [0]
      3976:	symbol=_ZN4llvm19SelectionDAGBuilder16visitInsertValueERKNS_15InsertValueInstE;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	symbol=_ZN4llvm19SelectionDAGBuilder16visitInsertValueERKNS_15InsertValueInstE;  lookup in file=/usr/lib/libdl.so.2 [0]
      3976:	symbol=_ZN4llvm19SelectionDAGBuilder16visitInsertValueERKNS_15InsertValueInstE;  lookup in file=/usr/lib/librt.so.1 [0]
      3976:	symbol=_ZN4llvm19SelectionDAGBuilder16visitInsertValueERKNS_15InsertValueInstE;  lookup in file=/usr/lib/libpthread.so.0 [0]
      3976:	symbol=_ZN4llvm19SelectionDAGBuilder16visitInsertValueERKNS_15InsertValueInstE;  lookup in file=/usr/lib/libc.so.6 [0]
      3976:	symbol=_ZN4llvm19SelectionDAGBuilder16visitInsertValueERKNS_15InsertValueInstE;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]
      3976:	binding file /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]: normal symbol `_ZN4llvm19SelectionDAGBuilder16visitInsertValueERKNS_15InsertValueInstE'
      3976:	symbol=jl_rethrow;  lookup in file=julia [0]
      3976:	symbol=jl_rethrow;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	binding file julia [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]: normal symbol `jl_rethrow'
      3976:	symbol=_ZNK4llvm8ZExtInst9cloneImplEv;  lookup in file=julia [0]
      3976:	symbol=_ZNK4llvm8ZExtInst9cloneImplEv;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	symbol=_ZNK4llvm8ZExtInst9cloneImplEv;  lookup in file=/usr/lib/libdl.so.2 [0]
      3976:	symbol=_ZNK4llvm8ZExtInst9cloneImplEv;  lookup in file=/usr/lib/librt.so.1 [0]
      3976:	symbol=_ZNK4llvm8ZExtInst9cloneImplEv;  lookup in file=/usr/lib/libpthread.so.0 [0]
      3976:	symbol=_ZNK4llvm8ZExtInst9cloneImplEv;  lookup in file=/usr/lib/libc.so.6 [0]
      3976:	symbol=_ZNK4llvm8ZExtInst9cloneImplEv;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]
      3976:	binding file /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]: normal symbol `_ZNK4llvm8ZExtInst9cloneImplEv'
      3976:	symbol=_ZN4llvm12SelectionDAG17getBoolExtOrTruncENS_7SDValueERKNS_5SDLocENS_3EVTES5_;  lookup in file=julia [0]
      3976:	symbol=_ZN4llvm12SelectionDAG17getBoolExtOrTruncENS_7SDValueERKNS_5SDLocENS_3EVTES5_;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	symbol=_ZN4llvm12SelectionDAG17getBoolExtOrTruncENS_7SDValueERKNS_5SDLocENS_3EVTES5_;  lookup in file=/usr/lib/libdl.so.2 [0]
      3976:	symbol=_ZN4llvm12SelectionDAG17getBoolExtOrTruncENS_7SDValueERKNS_5SDLocENS_3EVTES5_;  lookup in file=/usr/lib/librt.so.1 [0]
      3976:	symbol=_ZN4llvm12SelectionDAG17getBoolExtOrTruncENS_7SDValueERKNS_5SDLocENS_3EVTES5_;  lookup in file=/usr/lib/libpthread.so.0 [0]
      3976:	symbol=_ZN4llvm12SelectionDAG17getBoolExtOrTruncENS_7SDValueERKNS_5SDLocENS_3EVTES5_;  lookup in file=/usr/lib/libc.so.6 [0]
      3976:	symbol=_ZN4llvm12SelectionDAG17getBoolExtOrTruncENS_7SDValueERKNS_5SDLocENS_3EVTES5_;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]
      3976:	binding file /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]: normal symbol `_ZN4llvm12SelectionDAG17getBoolExtOrTruncENS_7SDValueERKNS_5SDLocENS_3EVTES5_'
      3976:	symbol=_ZN4llvm14TargetLowering15DAGCombinerInfo13AddToWorklistEPNS_6SDNodeE;  lookup in file=julia [0]
      3976:	symbol=_ZN4llvm14TargetLowering15DAGCombinerInfo13AddToWorklistEPNS_6SDNodeE;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	symbol=_ZN4llvm14TargetLowering15DAGCombinerInfo13AddToWorklistEPNS_6SDNodeE;  lookup in file=/usr/lib/libdl.so.2 [0]
      3976:	symbol=_ZN4llvm14TargetLowering15DAGCombinerInfo13AddToWorklistEPNS_6SDNodeE;  lookup in file=/usr/lib/librt.so.1 [0]
      3976:	symbol=_ZN4llvm14TargetLowering15DAGCombinerInfo13AddToWorklistEPNS_6SDNodeE;  lookup in file=/usr/lib/libpthread.so.0 [0]
      3976:	symbol=_ZN4llvm14TargetLowering15DAGCombinerInfo13AddToWorklistEPNS_6SDNodeE;  lookup in file=/usr/lib/libc.so.6 [0]
      3976:	symbol=_ZN4llvm14TargetLowering15DAGCombinerInfo13AddToWorklistEPNS_6SDNodeE;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]
      3976:	binding file /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-3.9.so [0]: normal symbol `_ZN4llvm14TargetLowering15DAGCombinerInfo13AddToWorklistEPNS_6SDNodeE'
      3976:	symbol=jl_string_ptr;  lookup in file=julia [0]
      3976:	symbol=jl_string_ptr;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	binding file julia [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]: normal symbol `jl_string_ptr'
      3976:	symbol=jl_symbol_n;  lookup in file=julia [0]
      3976:	symbol=jl_symbol_n;  lookup in file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]
      3976:	binding file julia [0] to /home/juliohm/Projects/julia-0.6/usr/bin/../lib/libjulia.so.0.6 [0]: normal symbol `jl_symbol_n'
-1001

When I do the same operation in Python, it seems to link to beignet's libcl.so file correctly:

opening file=/usr/lib/beignet//libcl.so [0]; direct_opencount=1
      4496:	
      4496:	symbol=clGetExtensionFunctionAddress;  lookup in file=/usr/lib/beignet//libcl.so [0]
      4496:	binding file /usr/lib/beignet//libcl.so [0] to /usr/lib/beignet//libcl.so [0]: normal symbol `clGetExtensionFunctionAddress'
      4496:	symbol=clIcdGetPlatformIDsKHR;  lookup in file=/usr/lib/beignet//libcl.so [0]
      4496:	binding file /usr/lib/beignet//libcl.so [0] to /usr/lib/beignet//libcl.so [0]: normal symbol `clIcdGetPlatformIDsKHR'
      4496:	symbol=clGetPlatformInfo;  lookup in file=/usr/lib/beignet//libcl.so [0]
      4496:	binding file /usr/lib/beignet//libcl.so [0] to /usr/lib/beignet//libcl.so [0]: normal symbol `clGetPlatformInfo'

If you look at Julia's output, it has the wrong libOpenCL.so that is installed system-wise for some reason. I didn't installed it explicitly as far as I remember. How come Julia finds it first before the Beignet one? Maybe a runtime path issue? I will try to put beignet in front of my libpath to see if anything changes.

@iblislin
Copy link

How come Julia finds it first before the Beignet one?

hmm, searching /usr/lib is done by default by Libdl.

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

@iblis17 I am just wondering why Python can find the correct Beignet library even though it is in a different path? Maybe they do in a more conservative way by querying clinfo or something?

@vchuravy
Copy link
Member

Julia loads libOpenCL.so which is the so called ICD loader, which allows you to have more than one platform installed.
Interestingly enough Python seems to link on the C-level against that library. It would be interesting to see which libOpenCL.so they actually link against.
You can reduce the output by using LD_DEBUG=libs.

You got an error code there:

-1001 | CL_PLATFORM_NOT_FOUND_KHR | clGetPlatform | No valid ICDs found

So something on your system is missconfigured. Are you on a debian system? If so you can try:

dpkg -S /usr/lib/libOpenCL.so

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

@vchuravy the error -1001 only happens when I try to run the call for the second time. The first call returns the ReadMemoryError.

I am on Arch Linux, this is what I get from Python and Julia with LD_DEBUG=libs

Python

      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_struct.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/math.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	find library=libopenblasp-r0-39a31c03.2.18.so [0]; searching
      5157:	 search path=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/tls/haswell/x86_64:/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/tls/haswell:/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/tls/x86_64:/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/tls:/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/haswell/x86_64:/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/haswell:/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/x86_64:/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs		(RPATH from file /home/juliohm/.local/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so)
      5157:	  trying file=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/tls/haswell/x86_64/libopenblasp-r0-39a31c03.2.18.so
      5157:	  trying file=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/tls/haswell/libopenblasp-r0-39a31c03.2.18.so
      5157:	  trying file=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/tls/x86_64/libopenblasp-r0-39a31c03.2.18.so
      5157:	  trying file=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/tls/libopenblasp-r0-39a31c03.2.18.so
      5157:	  trying file=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/haswell/x86_64/libopenblasp-r0-39a31c03.2.18.so
      5157:	  trying file=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/haswell/libopenblasp-r0-39a31c03.2.18.so
      5157:	  trying file=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/x86_64/libopenblasp-r0-39a31c03.2.18.so
      5157:	  trying file=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/libopenblasp-r0-39a31c03.2.18.so
      5157:	
      5157:	find library=libgfortran-ed201abd.so.3.0.0 [0]; searching
      5157:	 search path=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs		(RPATH from file /home/juliohm/.local/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so)
      5157:	  trying file=/home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/libgfortran-ed201abd.so.3.0.0
      5157:	
      5157:	
      5157:	calling init: /home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/libgfortran-ed201abd.so.3.0.0
      5157:	
      5157:	
      5157:	calling init: /home/juliohm/.local/lib/python3.6/site-packages/numpy/core/../.libs/libopenblasp-r0-39a31c03.2.18.so
      5157:	
      5157:	
      5157:	calling init: /home/juliohm/.local/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_datetime.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /home/juliohm/.local/lib/python3.6/site-packages/numpy/core/umath.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	find library=libffi.so.6 [0]; searching
      5157:	 search cache=/etc/ld.so.cache
      5157:	  trying file=/usr/lib/libffi.so.6
      5157:	
      5157:	
      5157:	calling init: /usr/lib/libffi.so.6
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_pickle.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	find library=libz.so.1 [0]; searching
      5157:	 search cache=/etc/ld.so.cache
      5157:	  trying file=/usr/lib/libz.so.1
      5157:	
      5157:	
      5157:	calling init: /usr/lib/libz.so.1
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/zlib.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	find library=libbz2.so.1.0 [0]; searching
      5157:	 search cache=/etc/ld.so.cache
      5157:	  trying file=/usr/lib/libbz2.so.1.0
      5157:	
      5157:	
      5157:	calling init: /usr/lib/libbz2.so.1.0
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_bz2.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	find library=liblzma.so.5 [0]; searching
      5157:	 search cache=/etc/ld.so.cache
      5157:	  trying file=/usr/lib/liblzma.so.5
      5157:	
      5157:	
      5157:	calling init: /usr/lib/liblzma.so.5
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_lzma.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/grp.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	find library=libcrypto.so.1.1 [0]; searching
      5157:	 search cache=/etc/ld.so.cache
      5157:	  trying file=/usr/lib/libcrypto.so.1.1
      5157:	
      5157:	
      5157:	calling init: /usr/lib/libcrypto.so.1.1
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_hashlib.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_blake2.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_sha3.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_bisect.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_random.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /home/juliohm/.local/lib/python3.6/site-packages/numpy/linalg/lapack_lite.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /home/juliohm/.local/lib/python3.6/site-packages/numpy/linalg/_umath_linalg.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /home/juliohm/.local/lib/python3.6/site-packages/numpy/fft/fftpack_lite.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /home/juliohm/.local/lib/python3.6/site-packages/numpy/random/mtrand.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_opcode.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	find library=libOpenCL.so.1 [0]; searching
      5157:	 search cache=/etc/ld.so.cache
      5157:	  trying file=/usr/lib/libOpenCL.so.1
      5157:	
      5157:	find library=libstdc++.so.6 [0]; searching
      5157:	 search cache=/etc/ld.so.cache
      5157:	  trying file=/usr/lib/libstdc++.so.6
      5157:	
      5157:	find library=libgcc_s.so.1 [0]; searching
      5157:	 search cache=/etc/ld.so.cache
      5157:	  trying file=/usr/lib/libgcc_s.so.1
      5157:	
      5157:	
      5157:	calling init: /usr/lib/libgcc_s.so.1
      5157:	
      5157:	
      5157:	calling init: /usr/lib/libstdc++.so.6
      5157:	
      5157:	
      5157:	calling init: /usr/lib/libOpenCL.so.1
      5157:	
      5157:	
      5157:	calling init: /home/juliohm/.local/lib/python3.6/site-packages/pyopencl/_cffi.abi3.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/site-packages/_cffi_backend.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/binascii.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_posixsubprocess.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/select.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	find library=libexpat.so.1 [0]; searching
      5157:	 search cache=/etc/ld.so.cache
      5157:	  trying file=/usr/lib/libexpat.so.1
      5157:	
      5157:	
      5157:	calling init: /usr/lib/libexpat.so.1
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/pyexpat.cpython-36m-x86_64-linux-gnu.so
      5157:	
      5157:	
      5157:	calling init: /usr/lib/python3.6/lib-dynload/_socket.cpython-36m-x86_64-linux-gnu.so
      5157:	

Julia

julia> ccall((:clGetPlatformIDs, :libOpenCL), Cint, (Cuint, Ptr{Cuint}, Ptr{UInt32}), 0, C_NULL, Ref{UInt32}())
      5179:	find library=libOpenCL [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib:/home/juliohm/Projects/julia-0.6/usr/bin/../lib/julia		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libOpenCL
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/julia/libOpenCL
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libOpenCL
      5179:	 search cache=/etc/ld.so.cache
      5179:	 search path=/usr/lib		(system search path)
      5179:	  trying file=/usr/lib/libOpenCL
      5179:	
      5179:	find library=libOpenCL.so [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib:/home/juliohm/Projects/julia-0.6/usr/bin/../lib/julia		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libOpenCL.so
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/julia/libOpenCL.so
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libOpenCL.so
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libOpenCL.so
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libOpenCL.so
      5179:	
      5179:	find library=libX11.so.6 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libX11.so.6
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libX11.so.6
      5179:	
      5179:	find library=libXext.so.6 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libXext.so.6
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libXext.so.6
      5179:	
      5179:	find library=libXfixes.so.3 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libXfixes.so.3
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libXfixes.so.3
      5179:	
      5179:	find library=libdrm_intel.so.1 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libdrm_intel.so.1
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libdrm_intel.so.1
      5179:	
      5179:	find library=libdrm.so.2 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libdrm.so.2
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libdrm.so.2
      5179:	
      5179:	find library=libGL.so.1 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libGL.so.1
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libGL.so.1
      5179:	
      5179:	find library=libEGL.so.1 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libEGL.so.1
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libEGL.so.1
      5179:	
      5179:	find library=libxcb.so.1 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libxcb.so.1
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libxcb.so.1
      5179:	
      5179:	find library=libpciaccess.so.0 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libpciaccess.so.0
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libpciaccess.so.0
      5179:	
      5179:	find library=libGLX.so.0 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libGLX.so.0
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libGLX.so.0
      5179:	
      5179:	find library=libGLdispatch.so.0 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libGLdispatch.so.0
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libGLdispatch.so.0
      5179:	
      5179:	find library=libXau.so.6 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libXau.so.6
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libXau.so.6
      5179:	
      5179:	find library=libXdmcp.so.6 [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libXdmcp.so.6
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libXdmcp.so.6
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libXdmcp.so.6
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libXau.so.6
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libGLdispatch.so.0
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libxcb.so.1
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libX11.so.6
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libXext.so.6
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libGLX.so.0
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libpciaccess.so.0
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libEGL.so.1
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libGL.so.1
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libdrm.so.2
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libdrm_intel.so.1
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libXfixes.so.3
      5179:	
      5179:	
      5179:	calling init: /usr/lib/beignet//libcl.so
      5179:	
      5179:	find library=libLLVM-5.0.so [0]; searching
      5179:	 search path=/home/juliohm/Projects/julia-0.6/usr/bin/../lib		(RPATH from file julia)
      5179:	  trying file=/home/juliohm/Projects/julia-0.6/usr/bin/../lib/libLLVM-5.0.so
      5179:	 search cache=/etc/ld.so.cache
      5179:	  trying file=/usr/lib/libLLVM-5.0.so
      5179:	
      5179:	find library=libffi.so.6 [0]; searching
      5179:	 search path=/usr/lib/../lib		(RPATH from file /usr/lib/libgfortran.so.4)
      5179:	  trying file=/usr/lib/../lib/libffi.so.6
      5179:	
      5179:	find library=libedit.so.0 [0]; searching
      5179:	 search path=/usr/lib/../lib		(RPATH from file /usr/lib/libgfortran.so.4)
      5179:	  trying file=/usr/lib/../lib/libedit.so.0
      5179:	
      5179:	find library=libtinfo.so.6 [0]; searching
      5179:	 search path=/usr/lib/../lib		(RPATH from file /usr/lib/libgfortran.so.4)
      5179:	  trying file=/usr/lib/../lib/libtinfo.so.6
      5179:	
      5179:	find library=libncursesw.so.6 [0]; searching
      5179:	 search path=/usr/lib/../lib		(RPATH from file /usr/lib/libgfortran.so.4)
      5179:	  trying file=/usr/lib/../lib/libncursesw.so.6
      5179:	
      5179:	
      5179:	calling init: /usr/lib/../lib/libtinfo.so.6
      5179:	
      5179:	
      5179:	calling init: /usr/lib/../lib/libncursesw.so.6
      5179:	
      5179:	
      5179:	calling init: /usr/lib/../lib/libedit.so.0
      5179:	
      5179:	
      5179:	calling init: /usr/lib/../lib/libffi.so.6
      5179:	
      5179:	
      5179:	calling init: /usr/lib/libLLVM-5.0.so
      5179:	
ERROR: ReadOnlyMemoryError()
Stacktrace:
 [1] anonymous at ./<missing>:?

@vchuravy
Copy link
Member

pacman -Qo /usr/lib/libOpenCL.so
/usr/lib/libOpenCL.so is owned by ocl-icd 2.2.11-1

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

This is what I get as well:

/usr/lib/libOpenCL.so is owned by ocl-icd 2.2.11-1

@vchuravy
Copy link
Member

So the difference that I am seeing is that. Julia uses /usr/lib/libOpenCL.so and Python is using /usr/lib/libOpenCL.so.1.
Which package does /usr/lib/libOpenCL.so.1 belong to?

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

I get the same owner:

pacman -Qo /usr/lib/libOpenCL.so.1
/usr/lib/libOpenCL.so.1 is owned by ocl-icd 2.2.11-1

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

Julia also links with /usr/lib/beignet/libcl.so beside libOpenCL.so is that expected?

@vchuravy
Copy link
Member

Yes it is, but I just noticed something else. libcl.so is loading llvm-5.0.so So we might be running into colliding LLVM versions again.

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

So Julia is building with LLVM 3.9, do you think if I rebuild Julia with system's LLVM 5.0 the issue can go away? I can also give a second try on Julia from Arch repo, it seems like following the release cycle well.

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

Giving it a try, building Julia again with USE_LLVM=1 in Make.inc

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

Oh yes, I remember, Julia doesn't support LLVM 5.0 right? What is a solution to this issue then? The build just failed.

@iblislin
Copy link

🤔 I don't think 0.6 can work with LLVM 5.0

and USE_LLVM=1 should be USE_SYSTEM_LLVM

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

Will Julia v0.7 (and v1.0) support more recent versions of LLVM? Or there is any alternative solution to this situation with OpenCL?

@iblislin
Copy link

on 0.7, LLVM 5.0 compiles, but not regularly tested. LLVM 6.0 doesn't compiled, IIRC.

@iblislin
Copy link

maybe you can

svn checkout svn://svn.archlinux.org/community/beignet

then, hack the cmake file to build with Julia's LLVM

@iblislin
Copy link

seems beignet supports LLVM 3.9

@juliohm
Copy link
Member Author

juliohm commented Jan 12, 2018

I will try build Julia v0.7 from master, thank you for the info.

@juliohm
Copy link
Member Author

juliohm commented Jan 13, 2018

I can confirm that the ccall works in Julia v0.7 built with LLVM 5.0. The OpenCL.jl package; however, is not ready for Julia v0.7.

@vchuravy
Copy link
Member

@juliohm can you try JuliaLang/julia#24890 with v0.7 (and LLVM 3.9.1)?

@juliohm
Copy link
Member Author

juliohm commented Jan 16, 2018

@vchuravy could you please confirm I just need to clone inhorton's fork, checkout his branch with the fix, and build it with LLVM 3.9?

@juliohm
Copy link
Member Author

juliohm commented Jan 16, 2018

Building it, I will report the success or not of the attempt soon it is done. Thanks.

@juliohm
Copy link
Member Author

juliohm commented Jan 16, 2018

I can confirm that cloning that branch and building it with no changes to the Make.inc has worked well with the ccall we have been trying. How can I make sure it is linking to LLVM 3.9?

@vchuravy
Copy link
Member

vchuravy commented Jan 16, 2018 via email

@juliohm
Copy link
Member Author

juliohm commented Jan 16, 2018

LLVM 3.9.1 confirmed:

Julia Version 0.7.0-DEV.3414
Commit 912ea1ea8* (2018-01-16 02:14 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
WORD_SIZE: 64
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
LAPACK: libopenblas64_
LIBM: libopenlibm
LLVM: libLLVM-3.9.1 (ORCJIT, skylake)
Environment:
JULIA_NUM_THREADS = 2

@ssfrr
Copy link

ssfrr commented Jan 28, 2018

I hit this exact issue (arch linux /w beignet). So glad I found the info here! I'm not quite ready to jump to 0.7, so I recompiled beignet against llvm3.9:

  1. clone beignet from https://github.com/intel/beignet
  2. install the llvm39 and clang39 packages
  3. do the cmake dance: mkdir build; cd build; cmake ..
  4. make
  5. If you want to be thorough, run the tests to make sure things worked out: make utest; . utests/setenv.sh; utests/utest_run (I actually got 1 failure here)
  6. Install with sudo make install

Now OpenCL.jl on 0.6.2 seems to work!

edit: I was getting warnings printed about "X server found. dri2 connection failed!", which appears to be a wayland issue - switching to Xorg solved the issue and I new/future versions of wayland shouldn't print the error.

@philtomson
Copy link

@ssfrr How do you tell cmake to use clang-3.9? I'm finding that after running cmake as you suggest above that when I look in the CMakeCache.txt file that clang-3.6 is being used.

@ssfrr
Copy link

ssfrr commented Apr 2, 2018

I'm not sure, I don't think I needed to do anything magical. Maybe I didn't have any other versions of clang installed when I ran the install? I'm not familiar with the beignet build process other than what I listed above, but perhaps you can just manually edit the paths within the ccmake tool to point to the right versions?

@philtomson
Copy link

Apparently you edit the build/CMakeCache.txt file and change all the '3.6' to '3.9' - I'm not entirely sure if that's the official cmake-way, but I think that worked. Not entirely sure how that CMakeCache.txt gets used by the generated Makefile but didn't dig very deep into that makefile. I'd guess the better way to do this would be to add something on the cmake commandline, but I'm not sure what that would be.

@juliohm
Copy link
Member Author

juliohm commented Aug 3, 2019

Closing the issue since the information here refers to Julia v0.6 and Julia v0.7 era.

The situation of OpenCL.jl in recent Julia versions is still problematic though.

@juliohm juliohm closed this as completed Aug 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants