[gentoo-soc] [GSoC] Weekly Report : May 27-June 2 -- BLAS/LAPACK Runtime Switch

[gentoo-soc] [GSoC] Weekly Report : May 27-June 2 -- BLAS/LAPACK Runtime Switch

[Mo Zhou]

Hi Benda,

During May 9 - May 27, I created a demonstration overlay
for our initial implimentation [3] (eselect + symlink).
The C++ demo program shipped by that git repo suggests
the draft implementation is working correctly.

During May 27 - June 2, I posted about the proposed
solution on Gentoo-dev to wait for comments. Unfortunately,
it seems that simply porting Debian's solution to Gentoo
is not the best choice, and there are people against it.

I redesigned the solution based on ld.so.conf:

1. sci-libs/lapack could expose all its shared objects
   and headers in the public dirs (i.e. /usr/include,
   /usr/$(get_libdir)/ ).

2. optimized implementations build additional libblas.so
   and liblapack.so objects with patched build systems.
   We hint ld.so with ld.so.conf via eselect, so any
   program linked against libblas.so and liblapack.so
   could make use of this mechanism.

The second point is different from our private discussion
in telegram. And different from my draft implementation,
the redesigned solution will introduce a new USE flag for
BLAS/LAPACK reverse dependencies to decide whether the
program should link against switchable BLAS/LAPACK or
specified BLAS/LAPACK.

Although there are negative responses against the first
proposal, we are not far from the original schedule --
this week I'm going to implement the redesigned solution
for sci-libs/lapack and ask Gentoo-dev for comments again.

Thanks,
Mo.

Reference:

1. Drawback of the eselect+symlink solution:
https://bugs.gentoo.org/632618
2. The eselect+ld.so.conf solution: https://bugs.gentoo.org/531842
3. My Overlay: https://github.com/cdluminate/my-overlay

Re: [gentoo-soc] [GSoC] Weekly Report : May 27-June 2 -- BLAS/LAPACK Runtime Switch

[Benda Xu]

Hi Mo,

Mo Zhou <lumin@debian.org> writes:

> I redesigned the solution based on ld.so.conf:
>
> 1. sci-libs/lapack could expose all its shared objects
>    and headers in the public dirs (i.e. /usr/include,
>    /usr/$(get_libdir)/ ).
>
> 2. optimized implementations build additional libblas.so
>    and liblapack.so objects with patched build systems.
>    We hint ld.so with ld.so.conf via eselect, so any
>    program linked against libblas.so and liblapack.so
>    could make use of this mechanism.
>
> The second point is different from our private discussion
> in telegram. And different from my draft implementation,
> the redesigned solution will introduce a new USE flag for
> BLAS/LAPACK reverse dependencies to decide whether the
> program should link against switchable BLAS/LAPACK or
> specified BLAS/LAPACK.

If the USE flag is set to be the specified BLAS/LAPACK, we pick a
default for the user, say OpenBLAS, is that correct?

> Although there are negative responses against the first
> proposal, we are not far from the original schedule --
> this week I'm going to implement the redesigned solution
> for sci-libs/lapack and ask Gentoo-dev for comments again.

Don't be discouraged by the comments.  The alternative opinions made us
understand our design choice more deeply than before.

Overall that looks good to me.  Thanks!

Yours,
Benda

Re: [gentoo-soc] [GSoC] Weekly Report : May 27-June 2 -- BLAS/LAPACK Runtime Switch

[Mo Zhou]

Hi Benda,

On 2019-06-04 08:04, Benda Xu wrote:
> 
> If the USE flag is set to be the specified BLAS/LAPACK, we pick a
> default for the user, say OpenBLAS, is that correct?

Yes. And OpenBLAS is a sane choice undoubtedly.
Felix Yan had confirmed that Archlinux uses OpenBLAS
by default. Specifically, I expect that:

 * USE="virtual-blas virtual-lapack" enables the switching feature.
 * USE="" means OpenBLAS will be used.

> Don't be discouraged by the comments.  The alternative opinions made us
> understand our design choice more deeply than before.

Agreed.

Best,
Mo

[gentoo-soc] [GSoC] Weekly Report : June 3-June 16 -- BLAS/LAPACK Runtime Switch

[Mo Zhou]

Hi list,

During the two weeks I finished the re-design of the
proposed BLAS/LAPACK runtime switching mechanism, and
actually implemented it. The initial design and
implementation was completely deprecated.

The details about the new ld.so.conf+eselect based
solution could be found from related discussions on
gentoo-dev. The code could be found here[1].

Specifically, the skeleton of the implementation
involves two customized eselect modules[2] and
the updated reference implementation[3]. They are
basically ready to merge but I still need to check
the help messages and manpages to assure correctness.

If no one is object to this solution, I'm going
to submit the first batch of PR on June 24~26.

According to the original schedule, the following
tasks should have been done before June 17:

1. update ebuild for reference blas/lapack and
   integrate the framework.
2. package BLIS and add it as a candidate

"1" was completed at the time of writing except
for a documentation check. "2" was finished and
it's actually working well. We are not lagging
behind the schedule despite the unexpected
redesign.

Best,
Mo.

[1] https://github.com/cdluminate/my-overlay
[2] https://github.com/cdluminate/my-overlay/tree/master/app-eselect
[3] https://github.com/cdluminate/my-overlay/tree/master/sci-libs/lapack

Re: [gentoo-soc] [GSoC] Weekly Report : June 3-June 16 -- BLAS/LAPACK Runtime Switch

[Benda Xu]

Hi Mo,

Mo Zhou <lumin@debian.org> writes:

> During the two weeks I finished the re-design of the
> proposed BLAS/LAPACK runtime switching mechanism, and
> actually implemented it. The initial design and
> implementation was completely deprecated.
>
> The details about the new ld.so.conf+eselect based
> solution could be found from related discussions on
> gentoo-dev. The code could be found here[1].
>
> Specifically, the skeleton of the implementation
> involves two customized eselect modules[2] and
> the updated reference implementation[3]. They are
> basically ready to merge but I still need to check
> the help messages and manpages to assure correctness.
>
> If no one is object to this solution, I'm going
> to submit the first batch of PR on June 24~26.
>
> According to the original schedule, the following
> tasks should have been done before June 17:
>
> 1. update ebuild for reference blas/lapack and
>    integrate the framework.
> 2. package BLIS and add it as a candidate
>
> "1" was completed at the time of writing except
> for a documentation check. "2" was finished and
> it's actually working well. We are not lagging
> behind the schedule despite the unexpected
> redesign.

The progress looks great. Keep on!

Yours,
Benda