[gentoo-dev] RFC: Rethinking virtual/opencl and eselect-opencl

[gentoo-dev] RFC: Rethinking virtual/opencl and eselect-opencl

[Marek Szuba]

[-- Attachment #1.1: Type: text/plain, Size: 5472 bytes --]

(with many thanks to everyone who mulled on this problem in #gentoo-dev
yesterday, mattst88 in particular)

So, yesterday's attempt to begin phasing support for 32-bit OpenCL out
of Gentoo (which, to remind everyone who may not have followed the
earlier discussion, would essentially acknowledge the status quo of most
OpenCL providers only supporting native amd64 ABI) has failed miserably:
the masking of abi_x86_32 in virtual/opencl on amd64 and of
virtual/opencl as a whole on x86 resulted in breakages in the dependency
graph.

On the one hand, there are several packages keyworded x86 which
unconditionally depend on virtual/opencl:
 - dev-libs/clhpp
 - dev-python/pyopencl
 - net-wireless/cpyrit-opencl
 - sci-libs/clblas
 - sci-libs/clblast
This is the easier part of the problem - seeing as most OpenCL providers
we now have in the tree (the exceptions being proprietary AMD and NVidia
drivers for legacy GPUs) do not support x86 anyway, it should be safe to
simply drop the keyword.

On the other, there is the much hairier issue of multilib-capable packages:
 - dev-libs/cl
 - app-emulation/crossover-bin
 - app-emulation/wine-staging
 - app-emulation/wine-vanilla
 - app-text/tesseract
 - media-libs/opencv
 - media-libs/x264
 - media-video/ffmpeg
for which the consensus of yesterday's discussion seems to be that
assuming the dropping of multilib or opencl support altogether is not an
option, we would have to have separate src_configure (at the very least)
stages for different ABIs.


It seems therefore that the easiest way of phasing out 32-bit OpenCL
would be for virtual/opencl to list a dummy provider that is both
keyworded x86 and multilib-capable, and provides both header files and
libraries reverse dependencies need to build successfully regardless of
what OpenCL devices, if any, are actually available. Fortunately there
already are packages in the tree which do just that - OpenCL ICD
loaders, i.e. dev-libs/ocl-icd (has stable keywords for both amd64 and
x86) and, since yesterday evening, dev-libs/opencl-icd-loader (the
official Khronos Group ICD loader, the first tagged version of which got
released in mid-March).

Then, between mattst88 and myself we have come to the conclusion that it
might in fact make sense to have virtual/opencl provide *only* an ICD
loader and merely inform the users what hardware-specific runtimes are
available. Advantages of this approach:
 - both ocl-icd and opencl-icd-loader are keyworded x86 (actually, I
realised this morning that I have made a mistake with the latter because
it is no longer up to developers to keyword things for x86 themselves -
but I *have* tested this in a 32-bit chroot) and multilib-capable so
they will keep the dependency tree consistent even if there are no
actual OpenCL runtimes capable of supporting abi_x86_32;
 - intel-neo, rocm-opencl-runtime, amdgpu-pro-opencl and mesa[opencl]
all actually *require* an IDC loader to work, intel-ocl-sdk (or at least
version 4 of it) works fine through a loader, and given
nvidia-drivers[uvm] ebuilds install an ICD file, I presume they are fine
with the loader as well;
 - not having to switch between OpenCL runtimes would allow us to phase
out eselect-opencl (similarly to how the introduction of an OpenGL
loader has allowed us to get rid of eselect-opengl), or at the very
least limit it to the management of OpenCL header files;
 - it would make it easier for users to enable out-of-tree runtimes,
e.g. manually installed whole AMDGPU-Pro stack or Beignet from an overlay;
 - the current VIDEO_CARDS-based way of selecting runtimes is quite
often misleading, namely:
    * intel-neo only supports Intel GPUs from Broadwell onwards rather
than everything served by the i965 driver;
    * amdgpu-pro-opencl is not compatible Vega 10 and newer GPUs, and to
make it worse it causes segfaults rather than simply ignore such GPUs;
    * nvidia-drivers[uvm] is actually several sets of drivers, with
different versions appropriate for different GPUs;
    * mesa[opencl] only supports some (old) Nvidia GPUs;
    * intel-ocl-sdk does not support non-Intel amd64 CPUs.
  Having the options presented to users in human-readable fashion will
IMHO work better.


Therefore, mattst88 and I (or to clarify: the write-up is mine but Matt
has contributed several important points to it) would like to propose
the following steps to take:

1. Introduce a new version of virtual/opencl with RDEPEND consisting
*only* of app-eselect/eselect-opencl and
dev-libs/ocl-icd[khronos-headers], with the list of actual runtimes
moved to a postinst message and expanded to explain what works where;

2. Have all ebuilds of OpenCL runtimes depend on virtual/opencl
(replacing the dependency on dev-libs/ocl-icd wherever it is present)
but do NOT call "eselect opencl" if they currently do;

3. Test if downstream applications are happy with the new unified OpenCL
headers required by the Khronos Group ICD loader and if they are, add
dev-libs/opencl-icd-loader to virtual/opencl as an alternative to ocl-icd;

4. Think what, if anything, we still want eselect-opencl to do, and in
the event of having decided to remove it or reduce its functionality
adjust some or all install paths in ICD-loader ebuilds to point directly
to /usr rather then subdirectories of /usr/$(get_libdir)/OpenCL/vendors.


Opinions? Comments? Flames?

-- 
Marecki




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] RFC: Rethinking virtual/opencl and eselect-opencl

[Marek Szuba]

[-- Attachment #1.1: Type: text/plain, Size: 1271 bytes --]

On 2020-04-02 15:31, Marek Szuba wrote:

> 3. Test if downstream applications are happy with the new unified OpenCL
> headers required by the Khronos Group ICD loader and if they are, add
> dev-libs/opencl-icd-loader to virtual/opencl as an alternative to ocl-icd;

Quick update on this: today I have attempted to rebuild and test various
OpenCL-dependent packages using the unified headers and
dev-libs/opencl-icd-loader. The good news is, they have all been
perfectly happy with them. The bad news is, I actually had to manually
symlink all contents
/usr/lib/OpenCL/vendors/opencl-icd-loader/include/CL/ into
/usr/include/CL/ for this to work because eselect-opencl contains a
hard-coded list of expected vendor header files.

In other words, it will be necessary to either teach eselect-opencl how
to handle the unified headers or make it possible for it to let the
contents of /usr/include/CL be. Personally I am leaning towards the
latter - it doesn't even handle legacy headers that well (it installs
several versions into /usr/lib/OpenCL/global but in the end offers no
way of switching between them), and in any case even its description
suggests its job is to switch between implementations rather than handle
global headers.

-- 
MS


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] RFC: Rethinking virtual/opencl and eselect-opencl

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1555 bytes --]

On Sun, 2020-04-05 at 22:13 +0100, Marek Szuba wrote:
> On 2020-04-02 15:31, Marek Szuba wrote:
> 
> > 3. Test if downstream applications are happy with the new unified OpenCL
> > headers required by the Khronos Group ICD loader and if they are, add
> > dev-libs/opencl-icd-loader to virtual/opencl as an alternative to ocl-icd;
> 
> Quick update on this: today I have attempted to rebuild and test various
> OpenCL-dependent packages using the unified headers and
> dev-libs/opencl-icd-loader. The good news is, they have all been
> perfectly happy with them. The bad news is, I actually had to manually
> symlink all contents
> /usr/lib/OpenCL/vendors/opencl-icd-loader/include/CL/ into
> /usr/include/CL/ for this to work because eselect-opencl contains a
> hard-coded list of expected vendor header files.
> 
> In other words, it will be necessary to either teach eselect-opencl how
> to handle the unified headers or make it possible for it to let the
> contents of /usr/include/CL be. Personally I am leaning towards the
> latter - it doesn't even handle legacy headers that well (it installs
> several versions into /usr/lib/OpenCL/global but in the end offers no
> way of switching between them), and in any case even its description
> suggests its job is to switch between implementations rather than handle
> global headers.

While at it, is there any chance to rewrite eselect-opencl so that it
stops writing into /usr and uses environment approach like eselect-
opengl does?

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] RFC: Rethinking virtual/opencl and eselect-opencl

[Marek Szuba]

[-- Attachment #1.1: Type: text/plain, Size: 949 bytes --]

On 2020-04-06 06:27, Michał Górny wrote:

> While at it, is there any chance to rewrite eselect-opencl so that it
> stops writing into /usr and uses environment approach like eselect-
> opengl does?

I think I'd rather just nuke eselect-opencl altogether and force the use
of an ICD loader for all implementations. Looks like it will be easier
than Matt and I thought too, touch wood:
 - as previously mentioned, most of the runtimes currently in the tree
actually REQUIRE using an ICD loader. Of the remaining two, I know for a
fact at least the latest version of intel-ocl-sdk works fine via a
loader so the only one that remains to be verified is
x11-drivers/nvidia-drivers[uvm];
 - likewise, I have confirmed that with the possible exception of
x11-drivers/nvidia-drivers[uvm], none of the presently available
runtimes install their own header files that would have to be symlinked
into place by eselect-opencl.

-- 
MS


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]