From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1Q7t7A-0007fd-MR for garchives@archives.gentoo.org; Thu, 07 Apr 2011 17:35:29 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 9307B1C09F; Thu, 7 Apr 2011 17:34:13 +0000 (UTC) Received: from mail-wy0-f181.google.com (mail-wy0-f181.google.com [74.125.82.181]) by pigeon.gentoo.org (Postfix) with ESMTP id 2D7381C09F for ; Thu, 7 Apr 2011 17:34:12 +0000 (UTC) Received: by wyi11 with SMTP id 11so3401874wyi.40 for ; Thu, 07 Apr 2011 10:34:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:reply-to:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to :content-type:content-transfer-encoding; bh=Nau2MIGaKAfqJ2DOysN2XvqAsafTMpBBKghrh78PE8w=; b=lbBLwbg8roZTrfyjjnYpSCaVf7+vtV5MZD2M/dqFcqcW3lcQ0rxUnlBrYGFjwLAB38 XMvMjEcCYV2RY1gQ+Ri8JFkaB/41wEYNqNZKCVb/ipod8iY1D/QEBwt6tsOCWI/JHVeF vuFbljgdY967yCfsiCGy9+c4y1zXuB9wgkOFk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:reply-to:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=MMzfc55ldvXDlRxNpaTz+AMa6NvBzoh+3yqZ5LB7wDCHf6imUJjQU4EAU8nTsXH7jt egSKRg1GYbd6fC/plhEk3JcU/6n5pB6v+nqwSydO9BZYk6WcnQvBrl3gekN9cNT1woMI 0gQ3ieFf2YXoXCmeRkrQOHjL4xG4SYwhTIe8I= Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-soc@lists.gentoo.org Reply-to: gentoo-soc@lists.gentoo.org MIME-Version: 1.0 Received: by 10.227.139.89 with SMTP id d25mr1271967wbu.58.1302197652191; Thu, 07 Apr 2011 10:34:12 -0700 (PDT) Sender: sebfabbro@gmail.com Received: by 10.216.243.143 with HTTP; Thu, 7 Apr 2011 10:34:12 -0700 (PDT) In-Reply-To: References: Date: Thu, 7 Apr 2011 10:34:12 -0700 X-Google-Sender-Auth: vsYzySW7aIUKD7fCitlUmdQBkXw Message-ID: Subject: Re: [gentoo-soc] Automated Numerical Libraries Benchmark suite From: =?UTF-8?Q?S=C3=A9bastien_Fabbro?= To: gentoo-soc@lists.gentoo.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Archives-Salt: X-Archives-Hash: 3088886864aec5d01ae777d6ffbb3e4e On Wed, Apr 6, 2011 at 09:10, =C3=93scar N=C3=A1jera wrote: > Hi, > I'm interested in participating in th eGoogle Summer of code this year, I > applied last year for the "Shape up linear algebra project", but maybe it > was to ambitious for being eligible, because now it's contents are separa= ted > in two different projects: Support for Fortran modules/libraries with > multiple compilers, and Automated numerical libraries benchmark suite. > > I'm interested as mentioned before in the later one, could I get some > additional advise for developing the application. I read that about the > requirement of actually writing a patch for the application, could I get > some advise too for that, are there some easy to fix bugs I could find. Dear Oscar, Thanks for considering applying again this year. I do not have access to your last year proposal anymore, but anyway here is what I am thinking for this year. You are obviously welcome to change anything to your own idea if you think it is worth it. Numerical libraries in Gentoo. The concerns are mostly linear algebra and fft libraries. The idea is for the user to switch between implementations of given official library API, and to provide an easy way to test correctness and speed for each of the testing implementation. We already have a framework [1], it has room for much improvement such as: o) allow the user to switch libraries, not just root o) make eselect blas/cblas/lapack robust to unpackaging o) possibly allow compiler versioning Some work is going on in my overlay [2] and look for the huge amount of bugs on bugs.gentoo.org concerning blas, lapack, atlas, mkl, eselect, etc... Ultimately, we want to allow the user to test correctness and speed by running benchmark suites, so that we could tune compilers, compiler flags and library implementations in an automated fashion. This is would be new in Gentoo. The idea is vague and very much up to the student. One simple possibility is to use the standard LAPACK test suite an= d HPL benchmarks. Then run the tests for every combination of fortran compilers, fortran flags, linear algebra libraries and select the most correct one, the fastest one, the most memory efficient one, etc... It could be done generically to do other kind of Gentoo benchmarks, and this is also good publicity for Gentoo. So the plan could be roughly: 1) develop robust and portable suite of tests for both for matrix/vector operations, solving algorithms, may be FFT routines if there is time. The tests included in the lapack tar ball from netlib and the HPL are very good starts. We also could test things for parallel and for FFTs, where you would have to look whether standard tests exist (this is part of your research for the proposal) and code them if not. If you know some C++, you can take a look at the Eigen [3] library where they have developed a decent framework to benchmark linear algebra packages. Then you would have to create ebuilds for your test benchmark tools. 2) automate the benchmarking for Gentoo based machines. You would have to write a few scripts that run the tests for each blas or lapack eselect implementation installed on the system and produce useful outputs/plots for the user to decide on which to select. For FFT it would have to be different because there is no "standard" API, but in portage we have only few FFT implementations (fftw, acml, mkl of what I know). 3) a user defined eselect module. I am working with Ulrich Mueller (ulm) to produce an easy mechanism to add new eselect modules. It would be nice to work with us to try to improve it even better, adding the possibility for a user to eselect a different module than the sysadmin does. Justin Bronder (jsbronder) had worked out something for MPI called empi [4], you could take a look and think how we could do it. It is probably more important to focus on 1) and 2), although 3) would be extremely useful for other parts of Gentoo and requires no numerical skills but operating system ones. As an example of a result of the project, this is how a user could test blas from netlib vs. blas from ACML using your project blas-benchmarks: $ sudo emerge blas-reference acml blas-benchmarks $ blas-benchmarks study results... $ eselect blas set To compare sets of compiler and flags, this is a bit trickier, you might have to think of a way to make a slot and an eselect module each time blas is installed with a different set of compiler and use flags. This is not clear why a user would need to keep all blas modules once an efficient one has been chosen (may be for cross-compiling). You could also just store the results in your blas-benchmarks, and re-install a blas with different flags, re-run the benchmarks and compare with the stored result. Finally you could also do something in the line of what has been done for GNAT [5], which would require to modify all the blas and lapack ebuilds but with a good eclass would not be a big deal. Anyway it is up to you to imagine a solution if needed. Note all of this is a very open design, if you cook up something else, for your proposal all the best! Think GPU could add some nice features! Find us on irc at #gentoo-science, write to the gentoo-soc mailing list or just email me for more. Also I could not find any Gentoo contribution from you. So if you do apply, make sure you read all the guidelines. We require the students to submit a patch/fix a bug. This is quite an exciting and challenging project. I hope this long email will help you. [1] http://www.gentoo.org/proj/en/science/blas-lapack.xml [2] http://git.overlays.gentoo.org/gitweb/?p=3Ddev/bicatali.git [3] http://eigen.tuxfamily.org/index.php?title=3DBenchmark [4] http://dev.gentoo.org/~jsbronder/empi.xml [5] http://www.gentoo.org/proj/en/prog_lang/ada/dev_reference.xml Best, -- Sebastien