[gentoo-soc] [GSoC] Weekly Report: July 1 - July 7 | BLAS/LAPACK Runtime Switch

[gentoo-soc] [GSoC] Weekly Report: July 1 - July 7 | BLAS/LAPACK Runtime Switch

[Mo Zhou]

Hi list,

This week I

1. finalized the sci-libs/openblas ebuild file
   and got it merged into ::gentoo.

2. bumped version for sci-libs/blis

3. tried to build all reverse dependencies of
   sci-libs/{blas,cblas,lapack}-reference and
   virtual/{blas,cblas,lapack} to scan for
   possible regression. Result is available
   here [1].

   For some important software I also ran
   unittests, for example numpy and octave.

4. fixed dependency information for two packages
   during the build test.

5. Wrote a preliminary documentation about the
   usage and the project [2]

Next I'm going to integrate intel's math kernel
library into this mechanism. By the mid-July
we would be able to present the document to
more audience and have the mechanism more widely
tested.

Maybe I'm a little bit ahead of schedule at this
point as all targets have almost been completed.
The fact that I'm maintaining a similar mechanism
for Debian isn't the main reason for this -- It's
the (unexpected) mechanism redesign that had
reduced the amount of work. This should be fine
as the new design is much better and smoother
compared to the original one.

I talked with Benda about the problem. We think
Gentoo prefox / portage + Debian integration
may be the next problem to look into.

[1] https://github.com/cdluminate/my-overlay/blob/master/checklist.txt
[2] https://github.com/cdluminate/my-overlay/blob/master/usage.md

Re: [gentoo-soc] Weekly Report: July 1 - July 7 | BLAS/LAPACK Runtime Switch

[Benda Xu]

Dear Mo,

Mo Zhou <lumin@debian.org> writes:

> Hi list,
>
> This week I
>
> 1. finalized the sci-libs/openblas ebuild file and got it merged into
> ::gentoo.
>
> 2. bumped version for sci-libs/blis
>
> 3. tried to build all reverse dependencies of
> sci-libs/{blas,cblas,lapack}-reference and virtual/{blas,cblas,lapack}
> to scan for possible regression. Result is available here [1].
>
>    For some important software I also ran
>    unittests, for example numpy and octave.
>
> 4. fixed dependency information for two packages during the build
> test.
>   
> 5. Wrote a preliminary documentation about the usage and the project
> [2]

They are just awesome!

> Next I'm going to integrate intel's math kernel library into this
> mechanism. By the mid-July we would be able to present the document to
> more audience and have the mechanism more widely tested.

Looks good to me.

> Maybe I'm a little bit ahead of schedule at this point as all targets
> have almost been completed.  The fact that I'm maintaining a similar
> mechanism for Debian isn't the main reason for this -- It's the
> (unexpected) mechanism redesign that had reduced the amount of
> work. This should be fine as the new design is much better and
> smoother compared to the original one.

This is a tremendous accomplishment!  Thank you for your hard work to
make it reality.

Please go ahead with the documentations and opening bug reports and
Github issues, to guide our users and developers to get familiar with
the switching framework.

> I talked with Benda about the problem. We think Gentoo prefox /
> portage + Debian integration may be the next problem to look into.

s/prefox/Prefix/

Would you please write up a project plan for the Gentoo portage / Debian
dpkg integration in order to revise our GSoC goals?  That might well be
natural next step of your SIMDebian[1].

Yours,
Benda

1. https://github.com/SIMDebian

Re: [gentoo-soc] [GSoC] Weekly Report: July 1 - July 7 | BLAS/LAPACK Runtime Switch

[Benda Xu]

Mo Zhou <lumin@debian.org> writes:

> Next I'm going to integrate intel's math kernel
> library into this mechanism. By the mid-July
> we would be able to present the document to
> more audience and have the mechanism more widely
> tested.

FYI, a relevant bug is at

  https://bugs.gentoo.org/672466

Cheers,
Benda

Re: [gentoo-soc] [GSoC] Weekly Report: July 1 - July 7 | BLAS/LAPACK Runtime Switch

[Mo Zhou]

On 2019-07-07 12:11, Benda Xu wrote:
> Mo Zhou <lumin@debian.org> writes:
> 
>> Next I'm going to integrate intel's math kernel
>> library into this mechanism. By the mid-July
>> we would be able to present the document to
>> more audience and have the mechanism more widely
>> tested.
> 
> FYI, a relevant bug is at
> 
>   https://bugs.gentoo.org/672466

I think it's worthwhile to invest a bit more time
to the MKL integration from ebuild side. After all
when comparing all the freely available CPU-based
BLAS/LAPACK implementations, MKL is always ridiculously
fast. (Although it's another story on AMD CPU).

Re: [gentoo-soc] [GSoC] Weekly Report: July 1 - July 7 | BLAS/LAPACK Runtime Switch

[Andrew Savchenko]

[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]

On Sun, 07 Jul 2019 05:18:49 -0700 Mo Zhou wrote:
> On 2019-07-07 12:11, Benda Xu wrote:
> > Mo Zhou <lumin@debian.org> writes:
> > 
> >> Next I'm going to integrate intel's math kernel
> >> library into this mechanism. By the mid-July
> >> we would be able to present the document to
> >> more audience and have the mechanism more widely
> >> tested.
> > 
> > FYI, a relevant bug is at
> > 
> >   https://bugs.gentoo.org/672466
> 
> I think it's worthwhile to invest a bit more time
> to the MKL integration from ebuild side. After all
> when comparing all the freely available CPU-based
> BLAS/LAPACK implementations, MKL is always ridiculously
> fast. (Although it's another story on AMD CPU).

Not always. I made tests on Gentoo-based HPC cluster and OpenBLAS
was a bit faster on LINPACK test. This "a bit" was about
0.50 ± 0.05% within 95% probability range. Not much, but
statistically significant. OpenBLAS was built with -march=native
mode. Though this result is about 6 years old now.

Anyway integrating MKL in the runtime switching scheme is a good
idea. Thank you for your work.

Best regards,
Andrew Savchenko

[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]