[gentoo-science] sage on gentoo -- only use system atlas

[gentoo-science] sage on gentoo -- only use system atlas

[P Purkayastha]

[-- Attachment #1: Type: text/plain, Size: 2344 bytes --]

Hi all,

  I need to have a personal installation of Sage to do development work.
However, I also want to reduce my upgrade times by installing a system
atlas.

  To do so, I added the science overlay and emerged
sci-libs/atlas[lapack,fortran,threads]. Thereafter, I added the environment
variable

SAGE_ATLAS_LIB=/usr/lib

and proceeded to compile sage in my home directory. Atlas fails with the
error that it can not find liblapack. The exact error message is given
below:

Cannot find liblapack.* in /usr/lib
Unable to find one of liblapack, libcblas, libatlas or libf77blas
in the directory None
Set SAGE_ATLAS_LIB to the directory containing liblapack, libcblas,
libatlas and libf77blas (either .a, .so, .dylib extensions) if
you wish to use existing ATLAS libraries. For more details, see
http://sagemath.org/doc/installation/source.html#environment-variables
Unset SAGE_ATLAS_LIB to build ATLAS from source.

I looked at the dependencies of sci-mathematics/sage in the sage-on-gentoo
overlay, and I see that virtual/cblas and virtual/lapack is pulled in by
sci-libs/fflas-ffpack. This ebuild is pulled in by sci-libs/linbox, and
finally linbox is pulled in by sage.

However, the following command only pulls in the virtual packages, which
means that I already have lapack installed, but probably under a different
library name.

~/tmp/sage-on-gentoo» emerge -pvq virtual/cblas virtual/lapack
[ebuild N ] virtual/cblas-2.0
[ebuild N ] virtual/lapack-3.4 USE="-doc"

Can anyone tell me which library name corresponds to liblapack.so? I can
then make a symlink and proceed with building my personal copy of sage.
Here is an output of equery files:

~/tmp/sage-on-gentoo» equery files atlas | grep '/usr/lib.*lapack'
/etc/env.d/alternatives/lapack/atlas-threads/usr/lib64/pkgconfig/lapack.pc
/etc/env.d/alternatives/lapack/atlas/usr/lib64/pkgconfig/lapack.pc
/usr/lib64/libatlclapack.so
/usr/lib64/libatlclapack.so.3
/usr/lib64/libatllapack.so
/usr/lib64/libatllapack.so.3
/usr/lib64/libptclapack.so
/usr/lib64/libptclapack.so.3
/usr/lib64/libptlapack.so
/usr/lib64/libptlapack.so.3
/usr/lib64/pkgconfig/atlas-clapack-threads.pc
/usr/lib64/pkgconfig/atlas-clapack.pc
/usr/lib64/pkgconfig/atlas-lapack-threads.pc
/usr/lib64/pkgconfig/atlas-lapack.pc

Thanks and regards,
    basu.

[-- Attachment #2: Type: text/html, Size: 2817 bytes --]

Re: [gentoo-science] sage on gentoo -- only use system atlas

[Burcin Erocal]

Hi,

On Wed, 21 Aug 2013 20:02:02 +0800
P Purkayastha <ppurka@gmail.com> wrote:

>   I need to have a personal installation of Sage to do development
> work. However, I also want to reduce my upgrade times by installing a
> system atlas.
> 
>   To do so, I added the science overlay and emerged
> sci-libs/atlas[lapack,fortran,threads]. Thereafter, I added the
> environment variable
> 
> SAGE_ATLAS_LIB=/usr/lib
> 
> and proceeded to compile sage in my home directory. Atlas fails with
> the error that it can not find liblapack. The exact error message is
> given below:
> 
> Cannot find liblapack.* in /usr/lib
> Unable to find one of liblapack, libcblas, libatlas or libf77blas
> in the directory None
> Set SAGE_ATLAS_LIB to the directory containing liblapack, libcblas,
> libatlas and libf77blas (either .a, .so, .dylib extensions) if
> you wish to use existing ATLAS libraries. For more details, see
> http://sagemath.org/doc/installation/source.html#environment-variables
> Unset SAGE_ATLAS_LIB to build ATLAS from source.

if you make Sage believe that the atlas package was installed properly,
the installation proceeds without trouble. So as a quick workaround,
just do

touch spkg/installed/atlas-<version_number>

I always do this on my system. It would be good to find out the proper
solution of course. :)


Cheers,
Burcin

Re: [gentoo-science] sage on gentoo -- only use system atlas

[François Bissey]

On 2013-08-22 00:19, Burcin Erocal wrote:
> Hi,
> 
> On Wed, 21 Aug 2013 20:02:02 +0800
> P Purkayastha <ppurka@gmail.com> wrote:
> 
>>   I need to have a personal installation of Sage to do development
>> work. However, I also want to reduce my upgrade times by installing a
>> system atlas.
>> 
>>   To do so, I added the science overlay and emerged
>> sci-libs/atlas[lapack,fortran,threads]. Thereafter, I added the
>> environment variable
>> 
>> SAGE_ATLAS_LIB=/usr/lib
>> 
>> and proceeded to compile sage in my home directory. Atlas fails with
>> the error that it can not find liblapack. The exact error message is
>> given below:
>> 
>> Cannot find liblapack.* in /usr/lib
>> Unable to find one of liblapack, libcblas, libatlas or libf77blas
>> in the directory None
>> Set SAGE_ATLAS_LIB to the directory containing liblapack, libcblas,
>> libatlas and libf77blas (either .a, .so, .dylib extensions) if
>> you wish to use existing ATLAS libraries. For more details, see
>> http://sagemath.org/doc/installation/source.html#environment-variables
>> Unset SAGE_ATLAS_LIB to build ATLAS from source.
> 
> if you make Sage believe that the atlas package was installed properly,
> the installation proceeds without trouble. So as a quick workaround,
> just do
> 
> touch spkg/installed/atlas-<version_number>
> 
> I always do this on my system. It would be good to find out the proper
> solution of course. :)
> 
The main problem here is that Gentoo support the installation of 
multiple BLAS/LAPACK at once.
For that reason there is no libraries on the system installed with the 
names:
libcblas.*
libf77blas.*
liblapack.*

My recommendation for you is to create a folder in your home directory 
and
link all the libraries you need with the appropriate name in a lib 
subfolder
and pass the parent directory to SAGE_ATLAS_LIB.

Francois

Re: [gentoo-science] sage on gentoo -- only use system atlas

[P Purkayastha]

[-- Attachment #1: Type: text/plain, Size: 2969 bytes --]

On Thu, Aug 22, 2013 at 8:39 AM, François Bissey <fbissey@slingshot.co.nz>wrote:

> On 2013-08-22 00:19, Burcin Erocal wrote:
>
>> Hi,
>>
>> On Wed, 21 Aug 2013 20:02:02 +0800
>> P Purkayastha <ppurka@gmail.com> wrote:
>>
>>    I need to have a personal installation of Sage to do development
>>> work. However, I also want to reduce my upgrade times by installing a
>>> system atlas.
>>>
>>>   To do so, I added the science overlay and emerged
>>> sci-libs/atlas[lapack,fortran,threads]. Thereafter, I added the
>>> environment variable
>>>
>>> SAGE_ATLAS_LIB=/usr/lib
>>>
>>> and proceeded to compile sage in my home directory. Atlas fails with
>>> the error that it can not find liblapack. The exact error message is
>>> given below:
>>>
>>> Cannot find liblapack.* in /usr/lib
>>> Unable to find one of liblapack, libcblas, libatlas or libf77blas
>>> in the directory None
>>> Set SAGE_ATLAS_LIB to the directory containing liblapack, libcblas,
>>> libatlas and libf77blas (either .a, .so, .dylib extensions) if
>>> you wish to use existing ATLAS libraries. For more details, see
>>> http://sagemath.org/doc/installation/source.html#environment-variables
>>> Unset SAGE_ATLAS_LIB to build ATLAS from source.
>>>
>>
>> if you make Sage believe that the atlas package was installed properly,
>> the installation proceeds without trouble. So as a quick workaround,
>> just do
>>
>> touch spkg/installed/atlas-<version_number>
>>
>> I always do this on my system. It would be good to find out the proper
>> solution of course. :)
>>
>
Thanks! This makes the compilation proceed further. FYI, in sage-git, the
corresponding directory is SAGE_ROOT/local/var/lib/sage/installed


One question though: can this trick be used with the other packages such as
R, givaro, etc. If this can be done, then it will be so much easier to
upgrade sage! I wouldn't have to rely on luck on whether the build finishes
in some finite time. :-)



>>  The main problem here is that Gentoo support the installation of
> multiple BLAS/LAPACK at once.
> For that reason there is no libraries on the system installed with the
> names:
> libcblas.*
> libf77blas.*
> liblapack.*
>
> My recommendation for you is to create a folder in your home directory and
> link all the libraries you need with the appropriate name in a lib
> subfolder
> and pass the parent directory to SAGE_ATLAS_LIB.
>
>  Francois


Actually I do have those libraries. Only missing one is liblapack.*. It is
probably one of the similarly named libraries, except I don't know which. I
don't want to find out by hit and trial given how finicky atlas is.

~» ll /usr/lib/{libcblas.so,libf77blas.so,libatlas.so}
lrwxrwxrwx 1 root root 13 Aug 20 19:39 /usr/lib/libatlas.so -> libatlas.so.3
lrwxrwxrwx 1 root root 14 Sep 15 2012 /usr/lib/libcblas.so -> libgslcblas.so
lrwxrwxrwx 1 root root 15 Aug 20 19:39 /usr/lib/libf77blas.so ->
libf77blas.so.3

[-- Attachment #2: Type: text/html, Size: 4247 bytes --]

Re: [gentoo-science] sage on gentoo -- only use system atlas

[Sébastien Fabbro]

On Thu, Aug 22, 2013 at 1:11 AM, P Purkayastha <ppurka@gmail.com> wrote:

> Actually I do have those libraries. Only missing one is liblapack.*. It is
> probably one of the similarly named libraries, except I don't know which. I
> don't want to find out by hit and trial given how finicky atlas is.
>

If you installed atlas with gentoo, you can see which library is
lapack with the atlas-lapack and atlas-lapack-threads pkg-config
files.
To avoid collision with lapack and cblas, the libcblas and liblapack
were renamed to libatlcblas and libatllapack. All the other libraries
kept their original names.

--
Sebastien

Re: [gentoo-science] sage on gentoo -- only use system atlas

[François Bissey]

On 2013-08-23 03:36, Sébastien Fabbro wrote:
> On Thu, Aug 22, 2013 at 1:11 AM, P Purkayastha <ppurka@gmail.com> 
> wrote:
> 
>> Actually I do have those libraries. Only missing one is liblapack.*. 
>> It is
>> probably one of the similarly named libraries, except I don't know 
>> which. I
>> don't want to find out by hit and trial given how finicky atlas is.
>> 
> 
> If you installed atlas with gentoo, you can see which library is
> lapack with the atlas-lapack and atlas-lapack-threads pkg-config
> files.
> To avoid collision with lapack and cblas, the libcblas and liblapack
> were renamed to libatlcblas and libatllapack. All the other libraries
> kept their original names.
> 
The existence of the links he mentioned makes me think he has left over
from the old virtual from the main tree.

Francois

Re: [gentoo-science] sage on gentoo -- only use system atlas

[P Purkayastha]

[-- Attachment #1: Type: text/plain, Size: 1331 bytes --]

Are you sure? They are all valid (though equery resolves the links):

~» equery b /usr/lib/{libcblas.so,libf77blas.so,libatlas.so}
* Searching for
/usr/lib/libcblas.so,/usr/lib/libf77blas.so,/usr/lib/libatlas.so ...
sci-libs/atlas-3.10.1-r1 (/usr/lib64/libatlas.so.3)
sci-libs/atlas-3.10.1-r1 (/usr/lib64/libf77blas.so.3)
sci-libs/gsl-1.14 (/usr/lib64/libgslcblas.so.0.0.0)



On Fri, Aug 23, 2013 at 5:17 AM, François Bissey <fbissey@slingshot.co.nz>wrote:

> On 2013-08-23 03:36, Sébastien Fabbro wrote:
>
>> On Thu, Aug 22, 2013 at 1:11 AM, P Purkayastha <ppurka@gmail.com> wrote:
>>
>>  Actually I do have those libraries. Only missing one is liblapack.*. It
>>> is
>>> probably one of the similarly named libraries, except I don't know
>>> which. I
>>> don't want to find out by hit and trial given how finicky atlas is.
>>>
>>>
>> If you installed atlas with gentoo, you can see which library is
>> lapack with the atlas-lapack and atlas-lapack-threads pkg-config
>> files.
>> To avoid collision with lapack and cblas, the libcblas and liblapack
>> were renamed to libatlcblas and libatllapack. All the other libraries
>> kept their original names.
>>
>>  The existence of the links he mentioned makes me think he has left over
> from the old virtual from the main tree.
>
> Francois
>
>

[-- Attachment #2: Type: text/html, Size: 2060 bytes --]

Re: [gentoo-science] sage on gentoo -- only use system atlas

[François Bissey]

On 2013-08-24 03:30, P Purkayastha wrote:
> Are you sure? They are all valid (though equery resolves the links):
> 
> ~» equery b /usr/lib/{libcblas.so,libf77blas.so,libatlas.so}
>  * Searching for
> /usr/lib/libcblas.so,/usr/lib/libf77blas.so,/usr/lib/libatlas.so ...
>  sci-libs/atlas-3.10.1-r1 (/usr/lib64/libatlas.so.3)
> sci-libs/atlas-3.10.1-r1 (/usr/lib64/libf77blas.so.3)
> sci-libs/gsl-1.14 (/usr/lib64/libgslcblas.so.0.0.0)
> 
atlas 3.10.1 and gsl 1.14 _at_the_same_time_? You have a mix of the old
virtual from the main tree and new libraries from the science overlay.

Portage will complain (yesterday).

When you decide to do the clean up be sure to post here so we can
make sure you are back in a sane state.

Francois

Re: [gentoo-science] sage on gentoo -- only use system atlas

[P Purkayastha]

[-- Attachment #1: Type: text/plain, Size: 3345 bytes --]

On Sat, Aug 24, 2013 at 6:57 PM, François Bissey <fbissey@slingshot.co.nz>wrote:

> On 2013-08-24 03:30, P Purkayastha wrote:
>
>> Are you sure? They are all valid (though equery resolves the links):
>>
>> ~» equery b /usr/lib/{libcblas.so,libf77blas.so,libatlas.so}
>>  * Searching for
>> /usr/lib/libcblas.so,/usr/lib/libf77blas.so,/usr/lib/libatlas.so ...
>>  sci-libs/atlas-3.10.1-r1 (/usr/lib64/libatlas.so.3)
>> sci-libs/atlas-3.10.1-r1 (/usr/lib64/libf77blas.so.3)
>> sci-libs/gsl-1.14 (/usr/lib64/libgslcblas.so.0.0.0)
>>
>>  atlas 3.10.1 and gsl 1.14 _at_the_same_time_? You have a mix of the old
> virtual from the main tree and new libraries from the science overlay.
>
> Portage will complain (yesterday).
>
> When you decide to do the clean up be sure to post here so we can
> make sure you are back in a sane state.
>
> Francois
>
>
Portage did complain today. I had to unmask eselect::gentoo and
gsl::science. Then I upgraded and it had the following changes to show

 ~» quietemerge -uDv world $QEMERGE
--------------------------------------------------- Pretended emerge
---------------------------------------------------

These are the packages that would be merged, in order:

Calculating dependencies .. ... .. ... .. .......... done!
[ebuild U ] sys-libs/timezone-data-2013c [2013b] USE="nls" 346 kB
[ebuild U ] media-libs/tiff-4.0.3-r4 [4.0.3-r2] USE="cxx jpeg lzma zlib
-jbig -static-libs" 0 kB
[ebuild U ] dev-libs/klibc-1.5.25 [1.5.23] USE="-debug" 499 kB
[ebuild U ] dev-lang/python-2.7.5-r2:2.7 [2.7.5:2.7] USE="gdbm ncurses
readline sqlite ssl threads (wide-unicode) xml -berkdb -build -doc
-examples -hardened -ipv6 -tk -wininst" 0 kB
[ebuild UD~] app-admin/eselect-1.3.6::gentoo [1.3.7-r100::science]
USE="-doc (-emacs%) (-vim-syntax%*)" 164 kB
[ebuild U ~] sys-apps/portage-2.2.1 [2.2.0] USE="(ipc) xattr -build -doc
-epydoc (-pypy2_0) -python2 -python3 (-selinux)" LINGUAS="-ru"
PYTHON_TARGETS="python2_7 (-pypy1_9) -pypy2_0 -python2_6 (-python3_1)
-python3_2 -python3_3 (-python3_4)" 881 kB
[ebuild U ~] sci-libs/gsl-1.16-r1::science [1.14::gentoo]
USE="-cblas-external -static-libs%" 3,452 kB
[ebuild N ] app-shells/bash-completion-1.3-r2 0 kB
[ebuild U ~] media-libs/fontconfig-2.10.93:1.0 [2.10.92:1.0] USE="-doc
-static-libs" ABI_X86="(64) -32 (-x32)" 1,491 kB

Total: 9 packages (7 upgrades, 1 downgrade, 1 new), Size of downloads:
6,830 kB

WARNING: One or more updates have been skipped due to a dependency conflict:

app-admin/eselect:0

(app-admin/eselect-1.3.7-r100::science, ebuild scheduled for merge)
conflicts with
  <app-admin/eselect-1.3.7 required by
(app-shells/bash-completion-1.3-r2::gentoo, ebuild scheduled for merge)


The links are still there, so nothing got hosed till now :-)

~» equery b /usr/lib/{libcblas.so,libf77blas.so,libatlas.so}
* Searching for
/usr/lib/libcblas.so,/usr/lib/libf77blas.so,/usr/lib/libatlas.so ...
sci-libs/atlas-3.10.1-r1 (/usr/lib64/libatlas.so.3)
sci-libs/atlas-3.10.1-r1 (/usr/lib64/libf77blas.so.3)
sci-libs/gsl-1.16-r1 (/usr/lib64/libgslcblas.so.0.0.0)

Thanks for the warning, though.  I am still trying to compile sage in my
home directory though. Currently stuck at scipy since it can't find ATLAS.
:-( I will try to trouble shoot and probably mail sage-devel about this.

[-- Attachment #2: Type: text/html, Size: 4249 bytes --]