[gentoo-embedded] slow compilation on NFS Root -> use an external USB hard disk ;-)

[gentoo-embedded] slow compilation on NFS Root -> use an external USB hard disk ;-)

[Christopher Friedt]

I just thought that I'd mention this, because it wasn't completely 
obvious to me at first, and maybe someone else who happens to google for 
this subject might come up with this result.

Using the stage3 gentoo arm-softfloat-linux-uclibc filesystem, I had for 
some time been natively compiling packages for my target system, with 
the root located over NFS.

Normally it wasn't so bad - sure it was quite slower than 
cross-compiling. But one thing I hated about cross compiling was 
randomly running into packages that use some sort of pkgconfig file that 
I didn't have present, or trying to run some native program in configure 
and coming up with an error.

When I got around to compiling the GNU Classpath (for use with jamvm, 
and stripped of all but the necessities), I found that compiling this 
app over an NFS root failed quite often because of out-of-memory errors. 
The process was killed outright. Then I read somewhere about creating a 
swap file on an NFS root, using losetup to create a loop device, and 
using swapon to load that swap into the kernel's VM. That turned out to 
be the worst idea I think I've ever had, because the overhead of using 
RPC with the NFS root was probably of O^2 complexity at that point, 
especially considering all of the swapping that had to be done with such 
a huge compile, using jikes of course.

Then I thought - hey, i have 2 external hard disks right in front of me 
that connect via USB - my board has USB - (lightbulb !). Now the jikes 
process is running at a staggering 50-70% as opposed to the 1-10% it was 
before. The overhead is considerably less with usb-storage, even with 
the swap file located on the external USB hard disk.

~/Chris
-- 
gentoo-embedded@gentoo.org mailing list

Re: [gentoo-embedded] slow compilation on NFS Root -> use an external USB hard disk ;-)

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1702 bytes --]

On Sunday 27 May 2007, Christopher Friedt wrote:
> Using the stage3 gentoo arm-softfloat-linux-uclibc filesystem, I had for
> some time been natively compiling packages for my target system, with
> the root located over NFS.
>
> Normally it wasn't so bad - sure it was quite slower than
> cross-compiling. But one thing I hated about cross compiling was
> randomly running into packages that use some sort of pkgconfig file that
> I didn't have present, or trying to run some native program in configure
> and coming up with an error.

ive found NFSv3 + TCP actually isnt so shabby ... you have to make sure to use 
those options in the kernel cmdline though as the default is NFSv2 + UDP

> When I got around to compiling the GNU Classpath (for use with jamvm,
> and stripped of all but the necessities), I found that compiling this
> app over an NFS root failed quite often because of out-of-memory errors.
> The process was killed outright. Then I read somewhere about creating a
> swap file on an NFS root, using losetup to create a loop device, and
> using swapon to load that swap into the kernel's VM. That turned out to
> be the worst idea I think I've ever had, because the overhead of using
> RPC with the NFS root was probably of O^2 complexity at that point,
> especially considering all of the swapping that had to be done with such
> a huge compile, using jikes of course.

this is the biggest problem and ive found not worth the hassle if possible ... 
doing network swap (whether it be via nbd or nfs or ...) is just not usable

if your machine doesnt have enough physical ram to compile w/out -pipe, then 
it's time to search for another option :)
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-embedded] slow compilation on NFS Root -> use an external USB hard disk ;-)

[KiberGus]

> if your machine doesnt have enough physical ram to compile w/out -pipe, then
> it's time to search for another option :)
> -mike

I've stuck in-to similar problem. You can use qemu in arm emulation
mode. It is slow enough, on my amd64 3200+ it is about 1.5 times, than
my asus A620 with 400 Mhz arv5, but you can get much more RAM. I have
512 phisical RAM and 200 can be esaily given to virtual machine.
-- 
gentoo-embedded@gentoo.org mailing list