[gentoo-soc] Project Glentoo: Status Report #4

[gentoo-soc] Project Glentoo: Status Report #4

[Robert Seaton]

Hi, all.

Project Glentoo is a GSoC project to combine a Plan 9 userspace with a
Gentoo base system.

Early in the week, I managed to hack together a work around for the
issue the #gentoo-releng team was facing last week, but we're still
waiting for the core team to come up with an official fix to the
baselayout issue (bug #371280). I ran into some more issues with the
catalyst build tool, but generating fresh stage1 - 3 cleared
everything up.

On Wednesday, I spent some time trying to get distcc working on an
Amazon EC2 high-compute instance to speed up generation of the process
of building a liveCD image, seeing as it takes quite some time to
generate a fresh image on my aging core2duo. I still haven't managed
to get that working, but it's something I plan to continue to look
into during the upcoming week.

By Thursday, I had successfully generated a new Glentoo liveCD image,
but ran into a problem: the image doesn't automatically log the user
in like the official install media, so it's not very useful. I
debugged for hours, eventually finding out via #gentoo-releng that
this is an issue that the x86 maintainer is aware of. He said that he
is going to ask someone (don't remember who) about leveraging PAM for
use with the official install media to auto-login and deprecating the
livecd-tools package.

While waiting for a fix for the login issue, I've started working on
X11 integration, which is what I'd originally planned to spend the
week on. I'll be unable to test if it works until the login issue is
fixed, but I figure that I can at least get it building while I wait.
Right now, I'm running into a build issue with Python -- something I
hope to fix in the upcoming week.

For this upcoming week, I plan on getting the xorg-x11 package built
and installed as part of the liveCD generation process. After I manage
to get that done, I'm going to either work on a project page for
Glentoo or write some documentation for Catalyst, seeing as the
current documentation is quite sparse. I also might work on getting
distcc via an Amazon EC2 instance running so that I can generate
images faster.

I think the project is progressing at a steady rate, albeit slowly.
Hopefully, once this login issue is cleared up, I will stop being
plagued by build issues. I can dream, can't I? Anyways, Glentoo is
still on schedule to be at the point I originally planned for midterm.

Cheers,
rps

Re: [gentoo-soc] Project Glentoo: Status Report #4

[Rich Freeman]

On Sun, Jun 19, 2011 at 7:42 PM, Robert Seaton <seatonr@dupage.edu> wrote:
> On Wednesday, I spent some time trying to get distcc working on an
> Amazon EC2 high-compute instance to speed up generation of the process
> of building a liveCD image, seeing as it takes quite some time to
> generate a fresh image on my aging core2duo. I still haven't managed
> to get that working, but it's something I plan to continue to look
> into during the upcoming week.

Out of curiosity - did you ever get this working in any practical way?
 I've found that DistCC to a remote server from a slower host hasn't
worked all that well.  In order to make up for network latency you
need to run VERY parallel builds (-j15/etc), and the problem is that
there is no way to tell make to do that just for files that use gcc.
The result is that as soon as you hit a package that uses quite a bit
of java/python/bash/etc you end up running at -j15 on your local
system and it just kills you - especially java.

If you do find a reliable way of getting DISTCC working with EC2 I for
one would love to see a quick write-up!

Oh, and likewise it sounds like Glentoo is becoming fairly usable, so
any quick howtos for those of us who have always wanted to tinker with
it would be wonderful.  I wouldn't worry about full documentation
until you're ready, but feel free to just toss up an illustrative
script to start from (download this, boot it, type this, etc).

Rich

Re: [gentoo-soc] Project Glentoo: Status Report #4

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 1307 bytes --]

On 10:25 Fri 15 Jul     , Rich Freeman wrote:
> On Sun, Jun 19, 2011 at 7:42 PM, Robert Seaton <seatonr@dupage.edu> wrote:
> > On Wednesday, I spent some time trying to get distcc working on an
> > Amazon EC2 high-compute instance to speed up generation of the process
> > of building a liveCD image, seeing as it takes quite some time to
> > generate a fresh image on my aging core2duo. I still haven't managed
> > to get that working, but it's something I plan to continue to look
> > into during the upcoming week.
> 
> Out of curiosity - did you ever get this working in any practical way?
>  I've found that DistCC to a remote server from a slower host hasn't
> worked all that well.  In order to make up for network latency you
> need to run VERY parallel builds (-j15/etc), and the problem is that
> there is no way to tell make to do that just for files that use gcc.
> The result is that as soon as you hit a package that uses quite a bit
> of java/python/bash/etc you end up running at -j15 on your local
> system and it just kills you - especially java.

There's a nice option -l (load average) that blocks new jobs from 
starting if the load is too high.

-- 
Thanks,
Donnie

Donnie Berkholz
Admin, Summer of Code
Gentoo Linux and X.Org
Blog: http://dberkholz.com


[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]