[gentoo-soc] Benchmarking suite - Report 6

[gentoo-soc] Benchmarking suite - Report 6

[Andrea Arteaga]

After my absence, the development is restarted very productively. The
work done in the last week:

* A very basic ebuild was done and the repository has now a
portage-tree structure, making the project installable using layman.
* Many lots of bugs were solved. Now the script is much more stable and usable.
* Some improvement in the output, which is now clearer. A summary of
the operations is printed at the beginning and the nesting of the
messages is correct.
* Everything is now logged. Emerge commands, suite compilations and
executions are logged into /var/log/benchmarks, which makes easy for
the user to understand what happens in case of error (or just for
awareness).
* blas, cblas and lapack are fully supported.
* A completely new module "blas_accuracy" has been added: this module
tests the implementations of blas for result correctness.
* The "clean code" work continues.

For blas, cblas and lapack libraries, I was using a modified version
of BTL as benchmarking suite. This approach is very good some such
libraries, but it won't be possible for everything. For example, in
order to test the numerical correctness of libraries, BTL is not the
right tool to use. Therefore, a different Python module which does not
rely on BTL has been introduced. Nevertheless, as much code as
possible is shared between BTL-modules and no-BTL-modules,
demostrating that the script now is sufficiently generic to support
different kinds of modules. This is important, as in the following I
plan to support libraries that require a very different approach from
blas-like libraries (e.g. fftw).

For the next week, I want to provide at least a basic support for:

* Introducing the support for fftw (maybe other fft libraries)
* Constructing a basic benchmarking suite for distributed numerical
libraries (PBLAS, ScaLAPACK, ...)
* Enhancing the ebuild so that the project honors the Gentoo standard

Best regards.
Andrea Arteaga

Re: [gentoo-soc] Benchmarking suite - Report 6

[Andrea Arteaga]

I forgot to say that an important feature that has been added to the
script is the capability to generate an HTML report that contains the
plots resulting from the tests.

Moreover, an objective for the next week is the implementation of an
accuracy test for LAPACK, and the support for CBLAS besides BLAS.

Best regards
Andrea Arteaga

2011/7/3 Andrea Arteaga <andyspiros@gmail.com>:
> After my absence, the development is restarted very productively. The
> work done in the last week:
>
> * A very basic ebuild was done and the repository has now a
> portage-tree structure, making the project installable using layman.
> * Many lots of bugs were solved. Now the script is much more stable and usable.
> * Some improvement in the output, which is now clearer. A summary of
> the operations is printed at the beginning and the nesting of the
> messages is correct.
> * Everything is now logged. Emerge commands, suite compilations and
> executions are logged into /var/log/benchmarks, which makes easy for
> the user to understand what happens in case of error (or just for
> awareness).
> * blas, cblas and lapack are fully supported.
> * A completely new module "blas_accuracy" has been added: this module
> tests the implementations of blas for result correctness.
> * The "clean code" work continues.
>
> For blas, cblas and lapack libraries, I was using a modified version
> of BTL as benchmarking suite. This approach is very good some such
> libraries, but it won't be possible for everything. For example, in
> order to test the numerical correctness of libraries, BTL is not the
> right tool to use. Therefore, a different Python module which does not
> rely on BTL has been introduced. Nevertheless, as much code as
> possible is shared between BTL-modules and no-BTL-modules,
> demostrating that the script now is sufficiently generic to support
> different kinds of modules. This is important, as in the following I
> plan to support libraries that require a very different approach from
> blas-like libraries (e.g. fftw).
>
> For the next week, I want to provide at least a basic support for:
>
> * Introducing the support for fftw (maybe other fft libraries)
> * Constructing a basic benchmarking suite for distributed numerical
> libraries (PBLAS, ScaLAPACK, ...)
> * Enhancing the ebuild so that the project honors the Gentoo standard
>
> Best regards.
> Andrea Arteaga
>