[gentoo-science] benchmarking

[gentoo-science] benchmarking

[fbissey]

Hi Andrea,

that's an interesting little project. I gave your suite a little try but it
broke down for me. I tried both as a normal user and root. I am also confused
as to where the results will be stored. I am guessing I didn't find any because
my test broke down.

Here is what happened:
python main.py cblas cblastests.in axpy axpby rot matrix_vector atv symv syr2
ger trisolve_vector matrix_matrix aat trisolve_matrix trmm
============================================================
The following tests will be run:
Test: atlas-gcc-n
 - Package: sci-libs/atlas-3.9.41-r0
 - Environment: CC="gcc" CFLAGS="-O2 -march=native"

Test: atlas-gcc
 - Package: sci-libs/atlas-3.9.41-r0
 - Environment: CC="gcc" CFLAGS="-O2"

Test: atlas-gcc-g
 - Package: sci-libs/atlas-3.9.41-r0
 - Environment: CC="gcc" CFLAGS="-O2 -floop-interchange -floop-strip-mine
-floop-block"

Test: open-gcc-n
 - Package: sci-libs/openblas-9999-r0
 - Environment: CC="gcc" CFLAGS="-O2 -march=native"

Test: gsl-gcc
 - Package: sci-libs/gsl-1.15-r1
 - Environment: CC="gcc" CFLAGS="-O2"

Test: open-gcc-ng
 - Package: sci-libs/openblas-9999-r0
 - Environment: CC="gcc" CFLAGS="-O2 -march=native -floop-interchange
-floop-strip-mine -floop-block"

Test: gsl-gcc-ng
 - Package: sci-libs/gsl-1.15-r1
 - Environment: CC="gcc" CFLAGS="-O2 -march=native -floop-interchange
-floop-strip-mine -floop-block"

Test: gsl-gcc-g
 - Package: sci-libs/gsl-1.15-r1
 - Environment: CC="gcc" CFLAGS="-O2 -floop-interchange -floop-strip-mine
-floop-block"

Test: open-gcc
 - Package: sci-libs/openblas-9999-r0
 - Environment: CC="gcc" CFLAGS="-O2"

Test: open-gcc-g
 - Package: sci-libs/openblas-9999-r0
 - Environment: CC="gcc" CFLAGS="-O2 -floop-interchange -floop-strip-mine
-floop-block"

Test: gsl-gcc-n
 - Package: sci-libs/gsl-1.15-r1
 - Environment: CC="gcc" CFLAGS="-O2 -march=native"

Test: atlas-gcc-ng
 - Package: sci-libs/atlas-3.9.41-r0
 - Environment: CC="gcc" CFLAGS="-O2 -march=native -floop-interchange
-floop-strip-mine -floop-block"

============================================================

BEGIN TEST 1 - atlas-gcc-n
-- Emerging package sci-libs/atlas-3.9.41-r0
-- Package sci-libs/atlas-3.9.41-r0 failed to emerge: tar xjf
/var/cache/benchmarks/packages//atlas-gcc-n/sci-libs/atlas-3.9.41-r0.tbz2 -C
/var/tmp/benchmarks/roots//atlas-gcc-n/

BEGIN TEST 2 - atlas-gcc
-- Emerging package sci-libs/atlas-3.9.41-r0
-- Package sci-libs/atlas-3.9.41-r0 failed to emerge: CC="gcc"
PKGDIR="/var/cache/benchmarks/packages//atlas-gcc/" CFLAGS="-O2" emerge
--ignore-default-opts -OB "=sci-libs/atlas-3.9.41"

BEGIN TEST 3 - atlas-gcc-g
-- Emerging package sci-libs/atlas-3.9.41-r0
-- Package sci-libs/atlas-3.9.41-r0 failed to emerge: CC="gcc"
PKGDIR="/var/cache/benchmarks/packages//atlas-gcc-g/" CFLAGS="-O2
-floop-interchange -floop-strip-mine -floop-block" emerge --ignore-default-opts
-OB "=sci-libs/atlas-3.9.41"

BEGIN TEST 4 - open-gcc-n
-- Emerging package sci-libs/openblas-9999-r0
-- Package sci-libs/openblas-9999-r0 failed to emerge: tar xjf
/var/cache/benchmarks/packages//open-gcc-n/sci-libs/openblas-9999-r0.tbz2 -C
/var/tmp/benchmarks/roots//open-gcc-n/

BEGIN TEST 5 - gsl-gcc
-- Emerging package sci-libs/gsl-1.15-r1
-- Package emerged
-- Testing gsl
Traceback (most recent call last):
  File "main.py", line 194, in <module>
    mod.run_test(root=root, impl=impl, testdir=testdir, env=test['env'])
  File "/home/work/fbissey/sandbox/blas-test/auto-numerical-bench/btlbase.py",
line 108, in run_test
    raise Exception("Compilation failed: " + compilecl)
Exception: Compilation failed: g++ -o
/var/tmp/benchmarks/tests//gsl-gcc/gsl/test btl//libs/BLAS/main.cpp
-Ibtl//actions -Ibtl//generic_bench -Ibtl//generic_bench/utils -Ibtl//libs/STL
-Ibtl//libs/BLAS -DNDEBUG -DCBLASNAME=cblas -DCBLAS_INTERFACE -lrt
-L/var/tmp/benchmarks/roots//gsl-gcc//usr/lib64/ -O2 -march=native
-fomit-frame-pointer -pipe -fno-strict-aliasing -floop-interchange
-floop-strip-mine -floop-block -lgslcblas

Re: [gentoo-science] benchmarking

[Andrea Arteaga]

Hello Francois.

Thank you for your interest and your time. Your feedback is very useful to me.
You have many emerge errors. Since the last version of the sccript
(this evening, UTC time) the emerge results will be logged, so you can
see why emerge fails. If you can, please let me know what is the
problem.

The last problem is more strange: the compilation of the benchmarking
suite fails. Where did you run the script from (shouldn't matter,
but...)? I will make this compilation to be logged, too.

The results are collected at the end. This does not happen if the
script crashes before, which is your case.

Thanks again.
Cheers
Andrea Arteaga