From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([69.77.167.62] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1LV9HQ-0005lU-Ut for garchives@archives.gentoo.org; Thu, 05 Feb 2009 18:48:53 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 19091E0401; Thu, 5 Feb 2009 18:48:52 +0000 (UTC) Received: from mail.uj.edu.pl (mail.uj.edu.pl [149.156.89.147]) by pigeon.gentoo.org (Postfix) with ESMTP id E093CE0401 for ; Thu, 5 Feb 2009 18:48:51 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-science@lists.gentoo.org Reply-to: gentoo-science@lists.gentoo.org MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; charset=windows-1252; format=flowed Received: from [149.156.87.132] by mail.uj.edu.pl (Sun Java(tm) System Messaging Server 6.3-5.02 (built Oct 12 2007; 32bit)) with ESMTPSA id <0KEL002R5W9E1S00@mail.uj.edu.pl> for gentoo-science@lists.gentoo.org; Thu, 05 Feb 2009 19:48:51 +0100 (CET) Message-id: <498B3492.80109@mol.uj.edu.pl> Date: Thu, 05 Feb 2009 19:48:50 +0100 From: Janusz Mordarski Organization: =?windows-1252?Q?Wydzia=3F_Biochemii=2C_Biofizyki_i_?= =?windows-1252?Q?Biotechnologii_=2C_Uniwersytet_Jagiellon=27ski?= User-Agent: Icedove 1.5.0.14eol (X11/20090105) To: gentoo-science@lists.gentoo.org Subject: Re: [gentoo-science] sys-cluster infiniband , mpi related soft. update request References: <497629A5.2060001@mol.uj.edu.pl> <200901220233.58997.alexxy@gentoo.org> <497A088E.20500@mol.uj.edu.pl> <200901252314.20524.alexxy@gentoo.org> <4982D6D8.9040408@mol.uj.edu.pl> In-reply-to: X-Archives-Salt: 60de9059-ca04-4ffc-afcf-5fe15ab6bab5 X-Archives-Hash: e4dce47ef07ccad63f35c3b190a883e2 Well ok i've made my own ebuild with openib-drivers , v 1.4 (in attachment), i tried it and it compiles except ehca flag (won't compile with it), if someone's interested, please check it and put into the overlay i have another question, i have big PERFORMANCE problems with infiniband on gentoo, no matter if i use kernel drivers or openib-drivers, here's what OSU latency benchmark says: first, OSU over Gigabit Ethernet (for comparison): # OSU MPI Latency Test v3.1.1 # Size Latency (us) 0 76.37 1 77.77 2 77.66 4 77.92 8 77.95 16 77.98 Infiniband - drivers from openib-drivers ebuild # OSU MPI Latency Test v3.1.1 # Size Latency (us) 0 50.23 1 52.10 2 51.99 4 51.62 8 51.79 16 52.17 Infiniband - drivers from kernel 2.6.27 - results are almost the same, except everywhere, where there is 5x (fifty) , it's 4x (fourty) Infiniband on CentOS (installed from RPM OFED) # OSU MPI Latency Test v3.1.1 # Size Latency (us) 0 1.65 1 1.82 2 1.81 4 1.72 8 1.60 16 1.61 so, the last one is perfect, Infiniband over IP works as expected in previous test my benchmarks suffered terrible performance loss over infiniband on gentoo (the same with OSU Bandwidth benchmark, on Gentoo: 500-750MB/s , on CentOS: 1400 MB/s) any suggestions? maybe some packages from this overlay are doing something wrong? -- Dept of Computational Biophysics & Bioinformatics, Faculty of Biochemistry, Biophysics and Biotechnology, Jagiellonian University, ul. Gronostajowa 7, 30-387 Krakow, Poland. Tel: (+48-12)-664-6380