[gentoo-embedded] personal compile-farm ?

[gentoo-embedded] personal compile-farm ?

[Christopher Friedt]

Hi everyone,

For a long time I've been considering various mini-itx form-factor
devices for multi-purpose use at home, as an HTPC, NAS, maybe messing
around with osx86, etc.

One particular use that I wanted to make of such a device would be to
use it as my own personal compile-farm, for various arm
cross-compilation builds, and binary packages for my laptop / netbook.

Recently, I was considering the Zotac IONITX-A. I would consider this
a fairly powerful, yet low-power device, with an Atom 330 dual-core
processor at 1.6 GHz and nVidia GPU ( Ion / 9400m ). I'm more than
certain that it would work well as an HTPC, but for a personal build
machine, I'd like to hear some feedback.

Does anyone on the list have a similar network-appliance that they use
for a personal compile-farm ? Neither of my aging x86 machines offer
any CPU features greater than sse2, and neither have multiple-cores.

For those who have a multi-core compile-farm at home, is there a
largely noticeable difference in speed?

If anyone does have a Zotac IONITX-A, how is the heat dissipation? Fan
or no fan?

Cheers,

Chris
On a slightly related note
PS: Alternatively, there has been some mention [1] of a dual-core
Ion-based device for the next AppleTV model or Mac Mini.

[1] http://www.tomshardware.com/news/apple-mac-nvidia-ion,6849.html

Re: [gentoo-embedded] personal compile-farm ?

[Karl Hiramoto]

Christopher Friedt wrote:
> Hi everyone,
>
> For a long time I've been considering various mini-itx form-factor
> devices for multi-purpose use at home, as an HTPC, NAS, maybe messing
> around with osx86, etc.
>
> One particular use that I wanted to make of such a device would be to
> use it as my own personal compile-farm, for various arm
> cross-compilation builds, and binary packages for my laptop / netbook.
>
> Recently, I was considering the Zotac IONITX-A. I would consider this
> a fairly powerful, yet low-power device, with an Atom 330 dual-core
> processor at 1.6 GHz and nVidia GPU ( Ion / 9400m ). I'm more than
> certain that it would work well as an HTPC, but for a personal build
> machine, I'd like to hear some feedback.
>
> Does anyone on the list have a similar network-appliance that they use
> for a personal compile-farm ? Neither of my aging x86 machines offer
> any CPU features greater than sse2, and neither have multiple-cores.
>
> For those who have a multi-core compile-farm at home, is there a
> largely noticeable difference in speed?
>
>   
I don't have an atom myself but from looking at the specs, for the same
amount of money, i'd think you'd be better of with a single machine with
lots of RAM and 4 to 8 cores with lots of cache for compiling.

Then multiple of those more powerful machines if you need.

-- 

--
Karl Hiramoto  http://karl.hiramoto.org/

Re: [gentoo-embedded] personal compile-farm ?

[Peter Stuge]

Karl Hiramoto wrote:
> i'd think you'd be better of with a single machine with lots of RAM
> and 4 to 8 cores with lots of cache for compiling.

I'd think so too. The Atom is kind of weak so I would definately go
for a Fam10 (Phenom) system. I didn't trust AMD hardware before, but
since expeirence with coreboot that all changed.

I think it's difficult for someone else to suggest a solution (I
realize you were mostly asking for samples of what others have) since
you have to factor in cooling requirement, power consumption,
physical size, cost and performance and find a combination that
you're willing to host in your home.


//Peter

Re: [gentoo-embedded] personal compile-farm ?

[Christopher Friedt]

Hi Karl,

On Sat, Sep 5, 2009 at 1:07 PM, Karl Hiramoto<karl@hiramoto.org> wrote:
>> For those who have a multi-core compile-farm at home, is there a
>> largely noticeable difference in speed?

Sorry, that should read 'difference in speed for builds'.

> I don't have an atom myself but from looking at the specs, for the same
> amount of money, i'd think you'd be better of with a single machine with
> lots of RAM and 4 to 8 cores with lots of cache for compiling.

Thanks for your reply.

I'm primarily interested in something that uses a smaller form-factor
for power, noise pollution, and space reasons, but 4 to 8 cores
probably doesn't fit in that class.

Most certainly I'll be installing the maximum amount of RAM possible -
i do practically all builds now using tmpfs, since RAM has gotten so
amazingly cheap and fast, so yes, RAM is key.

CCACHE also helps out considerably, but I haven't been using it lately
out of fear that it had some interference issues with ${CROSS}-emerge
.

Cheers,

C

Re: [gentoo-embedded] personal compile-farm ?

[wireless]

Christopher Friedt wrote:


> I'm primarily interested in something that uses a smaller form-factor
> for power, noise pollution, and space reasons, but 4 to 8 cores
> probably doesn't fit in that class.
> 
> Most certainly I'll be installing the maximum amount of RAM possible -
> i do practically all builds now using tmpfs, since RAM has gotten so
> amazingly cheap and fast, so yes, RAM is key.
> 
> CCACHE also helps out considerably, but I haven't been using it lately
> out of fear that it had some interference issues with ${CROSS}-emerge


Hello Chris,

I feel you pain..... I've longed for an embedded, low power system
that I can use to replace traditional servers with and use for a
variety of tasks. Maybe we have some commonality of need for such
a cost-effect board that we can use for a distributed cluster
for compiling and other needs?


My need is to also use the same low power mobo to stream video to,
as a local box attached to the back of a big monitor for video
surveillance. Sometimes that means running a Doz OS on the mobo,
or installing Doz software, via wine.......

Here's what I'm looking at:Portwell  WEBS-1010
http://www.linuxfordevices.com/c/a/News/Portwell-WEBS1010/?kc=LXDEMNL082609

Intel Atom-based PC that uses less than seven Watts and measures just
4.5 x 4.0 x 1.05 inches. The WEBS-1010 includes a miniSD socket and a
2.5-inch disk bay, HDMI video output, an IR receiver, and six USB
ports. Portwell says that the WEBS-1010 uses only six Watts in normal
operation, and seven Watts when playing 1080p H.264 video. "Absolutely
silent" when employing only its miniSD slot for storage, the device
can also accept a 2.5-inch, SATA hard disk drive internally.


My idea world would be one mobo both for the cluster, so I can grow it
as necessary, and the same mobo to distribute to anywhere my friends
need a monitor to watch their network resources or their remote video
surveillance streams......


I like AMD, they seem to have their priorities straight with Open
Source, but the Intel Atom does attract my attention. Ideally,
I'd prefer something based on the latest ARM chip (13?) or such
but, I'd prefer not going it alone in support.


So, I mention this, so you know that my needs(desires) are not very
far off from what you are seeking (a nicely support video chip(gpu)
my be more than you want. Certainly similar cores with max ram are
ideal.



hth,
James

Re: [gentoo-embedded] personal compile-farm ?

[Peter Stuge]

wireless wrote:
> I've longed for an embedded, low power system that I can use to
> replace traditional servers with and use for a variety of tasks.

Are you a hyperscale customer?

http://en.community.dell.com/blogs/direct2dell/archive/2009/05/19/dell-launches-quot-fortuna-quot-via-nano-based-server-for-hyperscale-customers.aspx

"20-29 Watts at full load" but still nice..


//Peter

Re: [gentoo-embedded] personal compile-farm ?

[Karl Hiramoto]

Christopher Friedt wrote:
> Hi Karl,
>
> On Sat, Sep 5, 2009 at 1:07 PM, Karl Hiramoto<karl@hiramoto.org> wrote:
>   
>>> For those who have a multi-core compile-farm at home, is there a
>>> largely noticeable difference in speed?
>>>       
>
> Sorry, that should read 'difference in speed for builds'.
>
>   
>> I don't have an atom myself but from looking at the specs, for the same
>> amount of money, i'd think you'd be better of with a single machine with
>> lots of RAM and 4 to 8 cores with lots of cache for compiling.
>>     
>
> Thanks for your reply.
>
> I'm primarily interested in something that uses a smaller form-factor
> for power, noise pollution, and space reasons, but 4 to 8 cores
> probably doesn't fit in that class.
>   

If you don't mind waiting a long time for your atom to compile, then i
suppose it's ok.  But compute wise, its pretty weak.

See the benchmarks a core 2 duo is many times faster than a atom:
http://www.tomshardware.com/reviews/intel-atom-efficiency,2069-8.html


If noise and space is a primary concern but you want more compute power
than an atom, then you could check out  Aopen and logic supply, they are
more expensive than atoms though, but you get what you pay for.

http://www.logicsupply.com/blog/2008/10/10/introducing-the-all-new-fanless-core-2-duo-system-the-gs-l10/
http://usa.aopen.com/

I've used some aopen products withe Core2 duos for HTPC and been
happy.   Some of them have integrated IR receiver, and HD out which
makes it nice.



--
Karl Hiramoto  http://karl.hiramoto.org/

Re: [gentoo-embedded] personal compile-farm ?

[Christopher Friedt]

On Sat, Sep 5, 2009 at 3:53 PM, Karl Hiramoto<karl@hiramoto.org> wrote:
> See the benchmarks a core 2 duo is many times faster than a atom:
> http://www.tomshardware.com/reviews/intel-atom-efficiency,2069-8.html

These benchmarks are for the Atom N230, as opposed to the dual-core
Atom N330, so not really a valid comparison. Clearly, any dual-core
chip will always outperform a comparable single-core chip.

For me, there's a some likelihood that even an Atom N230 will still
outperform the Pentium-M that I have in my laptop at the moment, so
for me, anything would really be an improvement. Having a dual-core
Atom for a build machine would mean 4 concurrent threads instead of 2.

I guess I should really just ask ...

Has anyone recently upgraded their Gentoo machine from a single-core
to a dual-core processor, and if so, was there a clearly noticeable
increase in build speed ?

Re: [gentoo-embedded] personal compile-farm ?

[Martin Guy]

On 9/5/09, Christopher Friedt <chrisfriedt@gmail.com> wrote:
>  >> For those who have a multi-core compile-farm at home, is there a
>  >> largely noticeable difference in speed?
>
> Sorry, that should read 'difference in speed for builds'.

I built a 12-cpu one (using 400MHz pentiums, the maximum MHz/price for
old junk on ebay at the time!) and got pretty much linear speedup
using distcc with 2 jobs on each CPU.

    M

Re: [gentoo-embedded] personal compile-farm ?

[jsyrytczyk]

On Saturday 05 September 2009 12:48:37 Christopher Friedt wrote:
> Hi everyone,
>
> For a long time I've been considering various mini-itx form-factor
> devices for multi-purpose use at home, as an HTPC, NAS, maybe messing
> around with osx86, etc.
>
> One particular use that I wanted to make of such a device would be to
> use it as my own personal compile-farm, for various arm
> cross-compilation builds, and binary packages for my laptop / netbook.
>
> Recently, I was considering the Zotac IONITX-A. I would consider this
> a fairly powerful, yet low-power device, with an Atom 330 dual-core
> processor at 1.6 GHz and nVidia GPU ( Ion / 9400m ). I'm more than
> certain that it would work well as an HTPC, but for a personal build
> machine, I'd like to hear some feedback.
>
> Does anyone on the list have a similar network-appliance that they use
> for a personal compile-farm ? Neither of my aging x86 machines offer
> any CPU features greater than sse2, and neither have multiple-cores.
>
> For those who have a multi-core compile-farm at home, is there a
> largely noticeable difference in speed?
>
> If anyone does have a Zotac IONITX-A, how is the heat dissipation? Fan
> or no fan?
>
> Cheers,
>
> Chris
> On a slightly related note
> PS: Alternatively, there has been some mention [1] of a dual-core
> Ion-based device for the next AppleTV model or Mac Mini.
>
> [1] http://www.tomshardware.com/news/apple-mac-nvidia-ion,6849.html

Hi, I got Atom 330 with this  motherboard sice  six months or so.
 
http://www.intel.com/products/desktop/motherboards/D945GCLF2-D945GCLF2D/D945GCLF2-D945GCLF2D-overview.htm

The machine runs all right, but the compilation times are similar to *very* 
outdated Celeron 1.7 GHz (which is one core only). The Atom goes very snappy 
on every other task it performs (software mirror, backup, rsync, remote NX 
station, email, iscsi, puppet, pulseaudio server etc.),

but 
   
compilation times *are* slow. 

I cannot give you reliable output of equery list | while read x; do genlop -t 
$x; done because compilation times were so large, I started to compile on 
another machine using distcc. If you want me to schedule a short example, 
write me.

Janusz.

Re: [gentoo-embedded] personal compile-farm ?

[Christopher Friedt]

Hi Janusz,

On Sat, Sep 5, 2009 at 11:31 PM,  <jsyrytczyk@uni.opole.pl> wrote:
> Hi, I got Atom 330 with this  motherboard sice  six months or so.
>
> http://www.intel.com/products/desktop/motherboards/D945GCLF2-D945GCLF2D/D945GCLF2-D945GCLF2D-overview.htm
>
> The machine runs all right, but the compilation times are similar to *very*
> outdated Celeron 1.7 GHz (which is one core only). The Atom goes very snappy
> on every other task it performs (software mirror, backup, rsync, remote NX
> station, email, iscsi, puppet, pulseaudio server etc.),
>
> but
>
> compilation times *are* slow.

Your reply has been the most relevent to date, having been the only
person who has an Atom 330 device for reference.

After reading about your compilation speeds, I did a bit of googling
for "intel core 2 duo vs intel atom 330", and found [1] which reveals
quite a lot. It would seem that all of the Atom 330 "slowdowns" are
caused by memory latency.

It seems, that the Atom (both the 230 and 330) were not designed to
use the blazingly fast FSB frequencies that all other modern Intel
processors use, which is likely the primary reason for (sub-par)
performance, and probably also the reason for their low power
consumption. Most of the 45nm Core-2 processors support FSB
frequencies around 1 or 1.3 GHz. On the other hand, the Atom 330 only
supports FSB frequencies of 533 MHz. In terms of silicon / FETs, high
clock speeds == high power leakage.

So essentially, if the clock speeds of a Duo and Atom core were the
same, then the Atom would require twice as much time as the Duo for
the the same amount of "work" (i.e. memory reads / writes). Assuming
that the Atom in-use power is about 1/2 of the duo, then both systems
consume the same amount of power for a "task", but the Atom takes
twice as long.

In reality, the Atom consumes over half of the Duo in-use power.
Therefore, power-efficiency ironically favours the Intel Core-2 Duo
rather than the Atom for computation-intensive applications. For
multimedia, I would say that the Atom is slightly more
power-efficient.

In my estimation, the lower in-use power of the Atom would be lost if
it used a 1.3 GHz FSB controller. Does anyone disagree?

Conclusion:

The main bottleneck on the Atom 330 is not the CPU frequency, but
rather the FSB frequency. Therefore, for a dedicated HTPC and / or NAS
device, the an Intel Atom 330 device is a good choice. For a
dedicated, low-power build machine, the Atom 330 is a bad choice for
performance, but a good choice if only moderate performance is
required.

For a box that is intended to be used for HTPC / NAS and also a
dedicated low-power build machine, an Atom 330 device is still a
decent choice, because at least it performs efficiently for 2 out of 3
functions, and it's unlikely (physically impossible?) that one will
find a comparable dual-core, low-power, fanless (and not liquid
cooled) device with a 1.3 GHz FSB.

So ... yea, I think I'll probably grab one of these ZOTAC boards
anyway, at least for having an HTPC. Using it as a dedicated build
machine would still be useful, even if the performance isn't
particularly great. In any event, it won't be building packages
constantly for my purposes, but only periodically.



Cheers,

Chris


[1] http://www.tomshardware.com/reviews/dual-core-atom-330,2141-6.html
[2] http://www.intel.com/products/processor/core2duo/specifications.htm
[3] http://processorfinder.intel.com/details.aspx?sSpec=SLG9Y

Re: [gentoo-embedded] personal compile-farm ?

[Ed W]

Christopher Friedt wrote:
> One particular use that I wanted to make of such a device would be to
> use it as my own personal compile-farm, for various arm
> cross-compilation builds, and binary packages for my laptop / netbook.
>   

Compiling seems to need only lots of ram and a fairly fast disk (some 
say a ram drive for cache can give you up to a 2x speedup - measure some 
samples though)

So I would have thought just buying a fairly cheap Core2 motherboard and 
chip, 8Gb+ ram, plus one of these low profile, plug into the board PSU's 
(pico psu say: http://www.mini-box.com/picoPSU-150-XT-102-power-kit ) 
and perhaps a small amount of onboard storage (usb drive?) would be 
optimal for price/cost.  You might not even bother buying a chassis, 
just leave it in the cardboard box it arrives in and ventilate...

Use nfs to one of your existing machines for main storage and plonk on a 
little local storage for booting the OS... 

I have a quad core machine here which runs a bunch of vservers, one of 
which is my development chroot, another is the file server, another is a 
web server, etc, etc.  I think the quad procs are too power hungry for 
this to have been a really good idea, but it's certainly pretty meaty 
and just sits idling in the corner with all my storage plugged into it 
and occasionally whiring into life to compile things.  It's headless and 
most of the time I work from a laptop as my main machine.

So right now I am going down the "one fairly beefy machine and 
virtualise all the others" route to minimise cost/power.  I think it's a 
good route and I have retired all the older machines which were sitting 
around the office.  I just thought I would need more processors than I 
do on a daily basis and whilst the quad core is meaty, on average if I 
were on a budget I think a dual core would be 80% of the processing 
power for 30% of the power budget...

Good luck

Ed W