Re: [gentoo-science] .la files harmful to BLAS/LAPACK structure

["François Bissey" <fbissey@slingshot.co.nz>]

> On 22/10/2020, at 1:23 PM, Aisha Tammy <gentoo.science@aisha.cc> wrote:
> 
> On 10/21/20 6:21 PM, François Bissey wrote:
>> Hi all,
>> So on and off removal of la files generated by libtool is discussed.
>> science packages have now a strong incentive to remove them everywhere
>> they can.
>> It all started when I was examining the output of ldd -r on libgiac.so
>> in the sage-on-gentoo overlay and I noticed that my selected openblas
>> wasn’t the one being picked up. Netlib’s blas and lapack was picked up
>> instead. There was still an instance of openblas via gsl.
>> I had to ask myself why that was. The ldso selection works by putting
>> a path to implementation library before /usr/lib{,64} in /etc/ld.so.conf.
>> That way when library are resolved at runtime your blas/lapack implementation
>> is chosen first.
>> This mechanism can be thrown off if the library/executable has any RUNPATH
>> set as those are resolved first. And there it was, in libgiac.so I had
>> RUNPATH=//usr/lib64/ [yes with a double / not a mistake].
>> I inspected giac’s code for a couple of days trying to see where it was coming from.
>> In the end it wasn’t from anything in the code. The final clue was that
>> when linking, not only did I have -Wl,-rpath //usr/lib64 coming out of
>> nowhere libnauty was linked as /usr/lib64/libnauty.so instead of -lnauty.
> Indeed, you are correct, this is not something we want.
> We are actively working to remove these wherever we find them.
> Unfortunately we are very short on volunteers and helpers.
> 

Yes, I should shoulder my bit in time. I have a whole backlog of stuff
for suitesparse. It is particularly annoying.


>> And indeed our new nauty ebuild, following debian’s lead, ship a library
>> and installs .la files. Removing those .la files resulted in libgiac.so
>> linking properly, the RUNPATH being removed and my choice of blas/lapack
>> being respected.
>> So what packages still ship .la files that would be of concern to us as they
>> interfere with blas/lapack?
>> * All the suitesparse ebuilds - that are not explicitly multibuild (messes up glpk).
>> * All the coinor ebuilds - probably with the same exceptions for mutlibuild.
>> * dev-libs/igraph (0.7.1-r2 at least, concerning as it links to blas/lapack)
> 
> I am maintaining igraph, the new versions don't install .la files, they use
> proper pkg-config .pc files.
> 
>> * libsemigroup [sage-on-gentoo]
>> * gap [sage-on-gentoo]
> This package is messed up.
> I am currently trying to upgrade this one for the ::science overlay
> but the outlook is bleak. I might just tell people to use the package
> in ::sage-on-gentoo, though currently my ebuild does manually remove
> the .la files.
> Theres just too many subpackages...
> 

Possible good news, latest gap has a package manager, we may be able to
do something similar to R with it. Although there are causing me grief
because they don’t follow the now standard flow of gap package having 
a GitHub repo. And when they do they sometime lack proper release.

>> * sci-libs/iml (concerning since it links to cblas)
>> * sys-cluster/mpich
> I am doubtful this package has any problems, the cluster team ebuilds are good
> and there are active people in the project.
> 

Combining MPI and blas/lapack or say scalapeck is common. mopish install
a .la file and I am concerned. I haven’t checked openmpi which is libtool
heavy.

>> * dev-libs/libltdl
>> There are more installed here that look less dangerous but we never know,
>> we should pursue elimination across the tree.
>> Anything using libtool to build and depending on any of these, will inherit
>> //usr/lib{,64}/ as a RUNPATH.
>> François
> 
> Thanks a lot for the information, unfortunately this is cumbersome work and
> ultimately we are very short on people.
> If you could open bugs on bugs.gentoo.org that would be helpful to keep
> track of these. Emails tend to be forgotten.

My time is unfortunately in rather short supply too. But I’ll do my best.

> 
> If you are interested, sending patches to fix this in a PR would be the best
> solution and we would really appreciate the help :)
> 
> In any case, there is little we can do in a short amount of time, this is in
> for the long haul.

I updated the sage-on-gentoo repo where it made sense but yes this is time consuming.
I wanted to raise awareness to that particular issue.

François