[gentoo-soc] cross_container_support - progress report #9

[gentoo-soc] cross_container_support - progress report #9

[Jing Huang]

Hi,

== Progress of last Week ==

In the last week,  my qemu-linux patches had been reviewed by the qemu
community.

I investigated how to config dynamic libraries in lxc container.:

1. mount host's /lib /usr/lib /usr/local/lib to lxc container

2. modify lxc container's ld.so.conf to link these libs.

3. ldconfig -Nf /etc/ld.so.conf to make settings available. (The
ld.so.cache will include arm libraries by default. How to ignore
them?)

I am still debugging this method.

Additionally, I found I must execute "qemu-static-i386  x86-program"
in lxc container to make it run. Is it right?


== Plans for this week ==

1. Still investigate dynamic libraries configure.

2. wait for vapier's help. If it is possible, would you offer me the
rough tech roadmap of patching crossdev and/or the toolchain eclass
you mentioned last letter?

Thanks.

Jing.

[gentoo-soc] Re: cross_container_support - progress report #9

[Luca Barbato]

On 07/31/2012 05:45 PM, Jing Huang wrote:
> Hi,
> 
> == Progress of last Week ==
> 
> In the last week,  my qemu-linux patches had been reviewed by the qemu
> community.
> 
> I investigated how to config dynamic libraries in lxc container.:
> 
> 1. mount host's /lib /usr/lib /usr/local/lib to lxc container
> 
> 2. modify lxc container's ld.so.conf to link these libs.
> 
> 3. ldconfig -Nf /etc/ld.so.conf to make settings available. (The
> ld.so.cache will include arm libraries by default. How to ignore
> them?)
> 
> I am still debugging this method.
> 
> Additionally, I found I must execute "qemu-static-i386  x86-program"
> in lxc container to make it run. Is it right?

No it is wrong, you should call the runtime linker.

> == Plans for this week ==
> 
> 1. Still investigate dynamic libraries configure.

It will take probably more time.

> 2. wait for vapier's help. If it is possible, would you offer me the
> rough tech roadmap of patching crossdev and/or the toolchain eclass
> you mentioned last letter?

First let's try to wrap up what we have now available and get more
people trying it =) That means document all the steps in the wiki and
get voluteers to try them.

lu

-- 

Luca Barbato
Gentoo/linux
http://dev.gentoo.org/~lu_zero

[gentoo-soc] Re: cross_container_support - progress report #9

[Jing Huang]

On Tue, Jul 31, 2012 at 7:30 PM, Luca Barbato <lu_zero@gentoo.org> wrote:
> On 07/31/2012 05:45 PM, Jing Huang wrote:
>> Hi,
>>
>> == Progress of last Week ==
>>
>> In the last week,  my qemu-linux patches had been reviewed by the qemu
>> community.
>>
>> I investigated how to config dynamic libraries in lxc container.:
>>
>> 1. mount host's /lib /usr/lib /usr/local/lib to lxc container
>>
>> 2. modify lxc container's ld.so.conf to link these libs.
>>
>> 3. ldconfig -Nf /etc/ld.so.conf to make settings available. (The
>> ld.so.cache will include arm libraries by default. How to ignore
>> them?)
>>
>> I am still debugging this method.
>>
>> Additionally, I found I must execute "qemu-static-i386  x86-program"
>> in lxc container to make it run. Is it right?
>
> No it is wrong, you should call the runtime linker.

When I execute x86-program(builded with dynamic libraries) in lxc
container, it shows:
"bash: x86-program: No such file or directory"

Obviously,  the container has not called the right dynamic
linker(ld-linux.so) to interpret the x86-program.
Maybe I could register this linker or the linker wrap to
/proc/sys/fs/binfmt_misc ?

>
>> == Plans for this week ==
>>
>> 1. Still investigate dynamic libraries configure.
>
> It will take probably more time.
>
>> 2. wait for vapier's help. If it is possible, would you offer me the
>> rough tech roadmap of patching crossdev and/or the toolchain eclass
>> you mentioned last letter?
>
> First let's try to wrap up what we have now available and get more
> people trying it =) That means document all the steps in the wiki and
> get voluteers to try them.

OK. I am doing it.

>
> lu
>
> --
>
> Luca Barbato
> Gentoo/linux
> http://dev.gentoo.org/~lu_zero
>

[gentoo-soc] Re: cross_container_support - progress report #9

[Jing Huang]

On Wed, Aug 1, 2012 at 5:01 PM, Jing Huang <jing.huang.pku@gmail.com> wrote:
> On Tue, Jul 31, 2012 at 7:30 PM, Luca Barbato <lu_zero@gentoo.org> wrote:
>> On 07/31/2012 05:45 PM, Jing Huang wrote:
>>> Hi,
>>>
>>> == Progress of last Week ==
>>>
>>> In the last week,  my qemu-linux patches had been reviewed by the qemu
>>> community.
>>>
>>> I investigated how to config dynamic libraries in lxc container.:
>>>
>>> 1. mount host's /lib /usr/lib /usr/local/lib to lxc container
>>>
>>> 2. modify lxc container's ld.so.conf to link these libs.
>>>
>>> 3. ldconfig -Nf /etc/ld.so.conf to make settings available. (The
>>> ld.so.cache will include arm libraries by default. How to ignore
>>> them?)
>>>
>>> I am still debugging this method.
>>>
>>> Additionally, I found I must execute "qemu-static-i386  x86-program"
>>> in lxc container to make it run. Is it right?
>>
>> No it is wrong, you should call the runtime linker.
>
> When I execute x86-program(builded with dynamic libraries) in lxc
> container, it shows:
> "bash: x86-program: No such file or directory"
>
> Obviously,  the container has not called the right dynamic
> linker(ld-linux.so) to interpret the x86-program.
> Maybe I could register this linker or the linker wrap to
> /proc/sys/fs/binfmt_misc ?

Hmm..  The root cause of this fault is:

./x86-program(built with dynamic libraries)
      -->load_elf_binary()
           -->open_exec("/lib/ld-linux.so.2");

lxc chroot lacks of /lib/ld-linux.so.2. This problem is a difficult
one. In my opinion, the following two steps are essential to solve it.

1. alter linker script to build x86-program using custom ld-linux.so.2
path ( /target/lib/ld-linux.so.2 ).

2. switch dynamic libraries to custom paths.
This point is relatively easy. I could use either LD_LIBRARY_PATH or
ld.so.conf.

"ldconfig /etc/ld.so.conf" is odd. I had altered ld.so.conf and
env.d/00basic to exclude original library path(/lib, /usr/lib,
/usr/local/lib), but the new generated cache file still include these
paths.
Unfortunately, linker will link libraries accroding to the ld cache
file unless declare  the DT_RPATH and LD_LIBRARY_PATH.

>
>>
>>> == Plans for this week ==
>>>
>>> 1. Still investigate dynamic libraries configure.
>>
>> It will take probably more time.
>>
>>> 2. wait for vapier's help. If it is possible, would you offer me the
>>> rough tech roadmap of patching crossdev and/or the toolchain eclass
>>> you mentioned last letter?
>>
>> First let's try to wrap up what we have now available and get more
>> people trying it =) That means document all the steps in the wiki and
>> get voluteers to try them.
>
> OK. I am doing it.
>
>>
>> lu
>>
>> --
>>
>> Luca Barbato
>> Gentoo/linux
>> http://dev.gentoo.org/~lu_zero
>>

[gentoo-soc] Re: cross_container_support - progress report #9

[Luca Barbato]

On 08/02/2012 05:23 PM, Jing Huang wrote:
> Hmm..  The root cause of this fault is:
> 
> ./x86-program(built with dynamic libraries)
>       -->load_elf_binary()
>            -->open_exec("/lib/ld-linux.so.2");
> 
> lxc chroot lacks of /lib/ld-linux.so.2. This problem is a difficult
> one. In my opinion, the following two steps are essential to solve it.
> 
> 1. alter linker script to build x86-program using custom ld-linux.so.2
> path ( /target/lib/ld-linux.so.2 ).
> 

Overriding it is an interesting task, for the time being try mixing
amd64 and arm binaries, they use /lib and /lib64 so it is fine.

> 2. switch dynamic libraries to custom paths.
> This point is relatively easy. I could use either LD_LIBRARY_PATH or
> ld.so.conf.
> 
> "ldconfig /etc/ld.so.conf" is odd. I had altered ld.so.conf and
> env.d/00basic to exclude original library path(/lib, /usr/lib,
> /usr/local/lib), but the new generated cache file still include these
> paths.
> Unfortunately, linker will link libraries accroding to the ld cache
> file unless declare  the DT_RPATH and LD_LIBRARY_PATH.

Yes, it is one of the issue to investigate.

lu