[gentoo-science] liblapack, libblas, numpy, and python virtualenvs

[gentoo-science] liblapack, libblas, numpy, and python virtualenvs

[Grant Goodyear]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

For reproducibility, I like to have a python virtualenv that has all of the
python libraries that I use, relying on portage just for python,
virtualenv, and virtualenvwrapper. Building numpy using "pip install numpy"
fails if I use the science-overlay versions of the reference lapack and
blas libraries, because pip can't find liblapack to link to it.

Right now I'm cheating and using the non-overlay versions, but I'm sure
that somebody has a better solution of how to help pip find the right
libraries.  Help?

Thanks,
g2boojum
-- 
Grant Goodyear
web: http://www.grantgoodyear.org
e-mail: grant@grantgoodyear.org

[-- Attachment #2: Type: text/html, Size: 884 bytes --]

Re: [gentoo-science] liblapack, libblas, numpy, and python virtualenvs

[jody]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

I had a similar problem in a different situation.
I also used a cheat: i created soft links so that the libraries could be
found ...

jody


On Thu, Apr 17, 2014 at 5:29 PM, Grant Goodyear <g2boojum@gmail.com> wrote:

> For reproducibility, I like to have a python virtualenv that has all of
> the python libraries that I use, relying on portage just for python,
> virtualenv, and virtualenvwrapper. Building numpy using "pip install numpy"
> fails if I use the science-overlay versions of the reference lapack and
> blas libraries, because pip can't find liblapack to link to it.
>
> Right now I'm cheating and using the non-overlay versions, but I'm sure
> that somebody has a better solution of how to help pip find the right
> libraries.  Help?
>
> Thanks,
> g2boojum
> --
> Grant Goodyear
> web: http://www.grantgoodyear.org
> e-mail: grant@grantgoodyear.org
>

[-- Attachment #2: Type: text/html, Size: 1589 bytes --]

Re: [gentoo-science] liblapack, libblas, numpy, and python virtualenvs

[Sébastien Fabbro]

[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]

On Thu, Apr 17, 2014 at 8:29 AM, Grant Goodyear <g2boojum@gmail.com> wrote:

> For reproducibility, I like to have a python virtualenv that has all of
> the python libraries that I use, relying on portage just for python,
> virtualenv, and virtualenvwrapper. Building numpy using "pip install numpy"
> fails if I use the science-overlay versions of the reference lapack and
> blas libraries, because pip can't find liblapack to link to it.
>
> Right now I'm cheating and using the non-overlay versions, but I'm sure
> that somebody has a better solution of how to help pip find the right
> libraries.  Help?
>

The problem lies in the broken design of numpy.distutils trying to do way
too much autodetection, and our ways to manage around it while keeping our
linear algebra modularity.
A possible workaround if you really want to use pip and the gentoo-science
overlay framework is to simply link your selected blas and lapack libraries
to /usr/lib/lib{blas,lapack}.so. I have not tested it, but I suspect this
should work for openblas and  {blas,lapack}-reference since they only have
one library to link.

Are you trying to do cross-distro reproducibility?

Sebastien

[-- Attachment #2: Type: text/html, Size: 1665 bytes --]