From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 76560138334 for ; Thu, 13 Jun 2019 07:15:18 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 83E2CE08F9; Thu, 13 Jun 2019 07:15:13 +0000 (UTC) Received: from mx1.riseup.net (mx1.riseup.net [198.252.153.129]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 18184E08AB for ; Thu, 13 Jun 2019 07:15:12 +0000 (UTC) Received: from capuchin.riseup.net (capuchin-pn.riseup.net [10.0.1.176]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.riseup.net (Postfix) with ESMTPS id CE7931A2222 for ; Thu, 13 Jun 2019 00:15:11 -0700 (PDT) X-Riseup-User-ID: 279C080F38198806AF0E5B1B1F2FC7133469B4D41A8714A67DE18B4C3C404F76 Received: from [127.0.0.1] (localhost [127.0.0.1]) by capuchin.riseup.net (Postfix) with ESMTPSA id A89991209BF for ; Thu, 13 Jun 2019 00:15:11 -0700 (PDT) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Date: Thu, 13 Jun 2019 00:15:11 -0700 From: Mo Zhou To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] RFC: BLAS and LAPACK runtime switching (Re-designed) In-Reply-To: <2d3636f5bd6a738f30a4ad2e697b1ddb@debian.org> References: <2d3636f5bd6a738f30a4ad2e697b1ddb@debian.org> Message-ID: X-Archives-Salt: 5550f8e9-f2be-4f4c-b934-690e1ddbfcdc X-Archives-Hash: 0177fea10e3ee06d85713101d17fc8d6 Hi Gentoo devs, I redesigned the solution for BLAS/LAPACK runtime switching. New solution is based on eselect+ld.so.conf . See following. > Goal > ---- > > * When a program is linked against libblas.so or liblapack.so > provided by any BLAS/LAPACK provider, the eselect-based solution > will allow user to switch the underlying library without recompiling > anything. Instead of manipulating symlinks, I wrote a dedicated eselect module for BLAS: https://github.com/cdluminate/my-overlay/blob/master/app-eselect/eselect-blas/files/blas.eselect-0.2 This implementation will generate a corresponding ld.so.conf file on switching and refresh ld.so cache. advantages: 1. not longer manipulates symlinks under package manager directory, see https://bugs.gentoo.org/531842 and https://bugs.gentoo.org/632624 2. we don't have to think about static lib and header switching like Debian does. > * When a program is linked against a specific implementation, e.g. > libmkl_rt.so, the solution doesn't break anything. This still holds with the new solution. > Solution > -------- > > Similar to Debian's update-alternatives mechanism, Gentoo's eselect > is good at dealing with drop-in replacements as well. My preliminary The redesigned solution totally diverted from Debian's solution. * sci-libs/lapack provides standard API and ABI for BLAS/CBLAS/LAPACK and LAPACKE. It provides a default set of libblas.so, libcblas.so, and liblapack.so . Reverse dependencies linked against the three libraries (reference blas) will take advantage of the runtime switching mechanism through USE="virtual-blas virtual-lapack". Reverse dependencies linked to specific implementations such as libopenblas.so won't be affected at all. * every non-standard BLAS/LAPACK implementations could be registered as alternatives via USE="virtual-blas virtual-lapack". Once the virtual-* flags are toggled, the ebuild file will build some extra shared objects with correct SONAME. For example: /usr/lib64/libblis.so.2 (SONAME=libblis.so.2, general purpose) /usr/lib64/blas/blis/libblas.so.3 (USE="virtual-blas", SONAME=libblas.so.3) /usr/lib64/blas/blis/libcblas.so.3 (USE="virtual-blas", SONAME=libcblas.so.3) * Reverse dependencies of BLAS/LAPACK could optionally provide the "virtual-blas virtual-lapack" USE flags. if use virtual-*: link against reference blas/lapack else: link against whatever the ebuild maintainer like and get rid of the switching mechanism > Proposed Changes > ---------------- > > 1. Deprecate sci-libs/{blas,cblas,lapack,lapacke}-reference from gentoo > main repo. They use exactly the same source tarball. It's not quite > helpful to package these components in a fine-grained manner. A > single > sci-libs/lapack package is enough. sci-libs/{blas,cblas,lapack,lapacke}::gentoo should be deprecated. They are based on exactly the same source tarball, and maintaining 4 ebuild files for a single tarball is not a good choice IHMO. Those old ebuild files seems to leverage the flexibility of upstream build system because it enables one to, for example, skip the reference blas build and use an existing optimized BLAS impelementation and hence introduce flexibility. That flexibility is hard to maintain and is not necessary anymore with the new runtime switching mechanism. That's why I propose to merge the 4 ebuild into a single one: sci-libs/lapack. We don't need to add the "reference" postfix because no upstream will loot the name "lapack". When talking about "lapack" it's always the reference implementation. > 2. Merge the "cblas" eselect unit into "blas" unit. It is potentially > harmful when "blas" and "cblas" point to different implementations. > That means "app-eselect/eselect-cblas" should be deprecated. eselect-cblas should be deprecated. That affects gsl because it is registered as an cblas alternative. gsl doesn't provide the standard BLAS (fortran) API+ABI so it will not be added as a runtime switching candidate. Does this redesinged solution look acceptable now? Best, Mo.