[gentoo-soc] Benchmarking suite - Report 9

[gentoo-soc] Benchmarking suite - Report 9

[Andrea Arteaga]

This is the report of the project "Automated benchmark suite for
numerical libraries in Gentoo" for the week 18-24 July.
This is also available on my blog [1].

== Project description ==

The project aims to develop a simple yet powerful automated system of
benchmarking for numerical libraries. The Gentoo software system
provides many implementations of widely used standards such as BLAS,
CBLAS, LAPACK, ScaLAPACK and some other numerical library such as
FFTW, MKL. The developed tools will aid the system maintener to choose
the best suited implementation with respect to the machine hardware
and to test the samer implementation or different ones with different
compilers, compiler versions and compile flags.

== Progrees during the week ==

This week the project has gained the following features:

* Work on the FFTW module. Now the following two-dimensional actions
are available:
** FFTW_2D_Forward_Measure
** FFTW_2D_Forward_Estimate
** FFTW_2D_Backward_Measure
** FFTW_2D_Backward_Estimate

* Work on the PBLAS/ScaLAPACK module:
** Parallel axpy
** Parallel matrix-vector multiply
** Parallel LU decomposition
** Parallel Cholesky decomposition
** Parallel QR decomposition
** Parallel SVD decomposition
** Parallel eigenvalues/eigenvectors computation

Regarding the ScaLAPACK actions, some more work is needed in order to
avoid singular or non-SPD matrices to be processed. This will be part
of the next week's work. The 2-dimensional FFTW actions work well, and
the module will also gain soon the 3-dimensional actions.

== Plan for the next week ==

The issue with ScaLAPACK raised another problem, too: every
implementation that is tested should receive the same input
matrices/vectors, in order to make the tests fair. Therefore a
decision has been taken: the matrices will be generated by a
deterministic random number generator -- probably a linear
congruential one -- and the tests will share the seeds. The seeds will
be taken from a set of seeds known to generate valid matrices (e.g.
SPD matrices for algorithms that require so). This will be an
important part of the work.

Another part regards the package dependencies. At this moment the
script does not handle package dependencies: it only installs the
desired packages into some specific root, but fails when the package
has dependencies which are not installed in the system. The script
will be adapted in order to also install the dependencies into the
same root and manage the environment variables.

A last part of the week plan regards the input configuration file.
When testing a dependent library (e.g. LAPACK, which depends on BLAS
and/or CBLAS), the user should be able to select as implementation for
the dependency (e.g. BLAS or CBLAS in this case) not only an installed
one, but also an implementataion that has been tested.


Best regards
Andrea Arteaga

[1] http://wp.me/pzWEm-86

Re: [gentoo-soc] Benchmarking suite - Report 9

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 839 bytes --]

On 00:43 Mon 25 Jul     , Andrea Arteaga wrote:
> The project aims to develop a simple yet powerful automated system of 
> benchmarking for numerical libraries. The Gentoo software system 
> provides many implementations of widely used standards such as BLAS, 
> CBLAS, LAPACK, ScaLAPACK and some other numerical library such as 
> FFTW, MKL. The developed tools will aid the system maintener to choose 
> the best suited implementation with respect to the machine hardware 
> and to test the samer implementation or different ones with different 
> compilers, compiler versions and compile flags.

How much of your project is Gentoo-specific? Could it be generalized for 
other distros with source builds?

-- 
Thanks,
Donnie

Donnie Berkholz
Admin, Summer of Code
Gentoo Linux and X.Org
Blog: http://dberkholz.com


[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-soc] Benchmarking suite - Report 9

[Andrea Arteaga]

[-- Attachment #1: Type: text/plain, Size: 1170 bytes --]

My project is much Gentoo-specific. It could be generalized to other systems
by using some common interfaces to e.g. package managers, but it would be:
* difficult to be done within my gsoc project
* not very useful: the project relies on the gentoo's capability to build
from sources each package with specific environment. This woul> On 00:43 Mon
25 Jul , Andrea Arteaga wrote:
>> The project aims to develop a simple yet powerful automated system of
>> benchmarking for numerical libraries. The Gentoo software system
>> provides many implementations of widely used standards such as BLAS,
>> CBLAS, LAPACK, ScaLAPACK and some other numerical library such as
>> FFTW, MKL. The developed tools will aid the system maintener to choose
>> the best suited implementation with respect to the machine hardware
>> and to test the samer implementation or different ones with different
>> compilers, compiler versions and compile flags.
>
> How much of your project is Gentoo-specific? Could it be generalized for
> other distros with source builds?
>
> --
> Thanks,
> Donnie
>
> Donnie Berkholz
> Admin, Summer of Code
> Gentoo Linux and X.Org
> Blog: http://dberkholz.com
>

[-- Attachment #2: Type: text/html, Size: 1409 bytes --]

Re: [gentoo-soc] Benchmarking suite - Report 9

[Sébastien Fabbro]

On Wed, Jul 27, 2011 at 11:06, Andrea Arteaga <andyspiros@gmail.com> wrote:
> My project is much Gentoo-specific. It could be generalized to other systems
> by using some common interfaces to e.g. package managers, but it would be:
> * difficult to be done within my gsoc project
>

There is a large part of the code which consists of extending BTL to
implement new benchmarks (LAPACK, ScaLAPACK, FFTW), and this is not
Gentoo specific. The plan is to submit all these changes upstream.

--
Sébastien