public inbox for gentoo-user@lists.gentoo.org
 help / color / mirror / Atom feed
* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-29 23:09 [gentoo-user] Sync'ing and compiling pkgs for multiple PCs Simon
@ 2008-07-29 22:38 ` Andrey Falko
  2008-07-29 23:46   ` Simon
  2008-07-30  8:15 ` Stroller
  1 sibling, 1 reply; 13+ messages in thread
From: Andrey Falko @ 2008-07-29 22:38 UTC (permalink / raw
  To: gentoo-user

On Tue, Jul 29, 2008 at 4:09 PM, Simon <turner25@gmail.com> wrote:
> Hi,
>  Below I explain my story, I show some difficulties and near the end ask a
> few questions, but last paragraph is my main question, comments on the rest
> is greatly appreciated.
>
>  I have 3 computers and they are all setup the same with the exception of
> the kernel that has different options for the youngest of my pcs.  I'm
> looking for a simple, easy and efficient way to keep them all up2date.
>
>  I want to avoid 3 --sync to the gentoo servers.  I have tried to setup one
> of my pc as central server and sync the others on it, but has the main
> disadvantage that the other PCs are dependant on it (ok, it's not so
> difficult to change make.conf).
>
>  Then, I would have to compile approximately 3 times the same software.  My
> computers are slow, I am poor, can't buy new hardware (FORGET IT).  So I
> tried turning on buildpkg and compile on my fastest PC.
>
>  I already use unison to sync my /home directory through ssh, i use it
> because it is simple and does exactly what I need out of the box.  (Compared
> to rsyncd, where you need to setup a server, setup the firewall, what about
> security?...).
>  Anyway, I decided to sync /usr/portage with unison as well, which worked
> fine and I had the binpkgs on the other pc.  When I did an `emerge -k -uDN
> world` I found that many packages still had to be built.  I extracted the
> list of packages and versions, formated properly and emerged them all on my
> fastest pc, ran through the same routine and at the end I had to recompile
> just a few packages that I let run (nothing like gcc or glibc, hehe).
>
>  Now, with my "unison technique" I believe the only flaw was that I didn't
> have the list of all packages on all my PCs and I wonder if there is a way
> to generate such a list, similar to what `emerge -vp -uDN world` gives but
> in a format ready for emerge, so I can dump the pkg list for the 3 pc in 3
> files and run emerge on those 3 files.  I would compile them -1, which, as I
> understand it would upgrade my system (upgrade but not change world entries)
> and install unneeded packages around.  I would then use --depclean which
> would remove the unneeded packages cleanly and finally revdep-rebuild.  I
> would be left with my fastest pc's world up2date + all binpkgs of all 3 pcs.
>
>  The advantage of my "unison technique" is it's transparent to any 3 pc, if
> a pkg is not there, it downloads the distfile from the internet and compiles
> it, completely independant.  Just doing the --sync requires a little bit of
> consideration.
>
>  I'm wondering if there is (i'm sure there is) a better practice in the
> compilation of binpkgs for use by multiple computers, so that none of the
> other computers will compile (quick update for them) and that the compiling
> computer can compile all needed pkgs in one run preferably.
>
> As usual, if you can answer my question directly on the mailing list that'd
> be nice, if you could give URLs to some documentation that answers my
> question that'd be very nice too!
>
> Thanks,  Simon
>
>

I don't have time right now to answer all of your questions, but in
terms of having all machines have the same list of installed packages,
all you probably need is /var/lib/portage/world . This lists all of
the packages that you emerged. I.e. those packages not brought in via
dependencies. If you have the same USE flags set on all machines, then
you should not see the need for new dependencies to be brought in on
any of the non-compile machines.



^ permalink raw reply	[flat|nested] 13+ messages in thread

* [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
@ 2008-07-29 23:09 Simon
  2008-07-29 22:38 ` Andrey Falko
  2008-07-30  8:15 ` Stroller
  0 siblings, 2 replies; 13+ messages in thread
From: Simon @ 2008-07-29 23:09 UTC (permalink / raw
  To: gentoo-user

Hi,
   Below I explain my story, I show some difficulties and near the end ask a few 
questions, but last paragraph is my main question, comments on the rest is 
greatly appreciated.

   I have 3 computers and they are all setup the same with the exception of the 
kernel that has different options for the youngest of my pcs.  I'm looking for a 
simple, easy and efficient way to keep them all up2date.

   I want to avoid 3 --sync to the gentoo servers.  I have tried to setup one of 
my pc as central server and sync the others on it, but has the main disadvantage 
that the other PCs are dependant on it (ok, it's not so difficult to change 
make.conf).

   Then, I would have to compile approximately 3 times the same software.  My 
computers are slow, I am poor, can't buy new hardware (FORGET IT).  So I tried 
turning on buildpkg and compile on my fastest PC.

   I already use unison to sync my /home directory through ssh, i use it because 
it is simple and does exactly what I need out of the box.  (Compared to rsyncd, 
where you need to setup a server, setup the firewall, what about security?...).
   Anyway, I decided to sync /usr/portage with unison as well, which worked fine 
and I had the binpkgs on the other pc.  When I did an `emerge -k -uDN world` I 
found that many packages still had to be built.  I extracted the list of 
packages and versions, formated properly and emerged them all on my fastest pc, 
ran through the same routine and at the end I had to recompile just a few 
packages that I let run (nothing like gcc or glibc, hehe).

   Now, with my "unison technique" I believe the only flaw was that I didn't 
have the list of all packages on all my PCs and I wonder if there is a way to 
generate such a list, similar to what `emerge -vp -uDN world` gives but in a 
format ready for emerge, so I can dump the pkg list for the 3 pc in 3 files and 
run emerge on those 3 files.  I would compile them -1, which, as I understand it 
would upgrade my system (upgrade but not change world entries) and install 
unneeded packages around.  I would then use --depclean which would remove the 
unneeded packages cleanly and finally revdep-rebuild.  I would be left with my 
fastest pc's world up2date + all binpkgs of all 3 pcs.

   The advantage of my "unison technique" is it's transparent to any 3 pc, if a 
pkg is not there, it downloads the distfile from the internet and compiles it, 
completely independant.  Just doing the --sync requires a little bit of 
consideration.

   I'm wondering if there is (i'm sure there is) a better practice in the 
compilation of binpkgs for use by multiple computers, so that none of the other 
computers will compile (quick update for them) and that the compiling computer 
can compile all needed pkgs in one run preferably.

As usual, if you can answer my question directly on the mailing list that'd be 
nice, if you could give URLs to some documentation that answers my question 
that'd be very nice too!

Thanks,  Simon



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-29 22:38 ` Andrey Falko
@ 2008-07-29 23:46   ` Simon
  0 siblings, 0 replies; 13+ messages in thread
From: Simon @ 2008-07-29 23:46 UTC (permalink / raw
  To: gentoo-user

> I don't have time right now to answer all of your questions, but in
> terms of having all machines have the same list of installed packages,
> all you probably need is /var/lib/portage/world . This lists all of
> the packages that you emerged. I.e. those packages not brought in via
> dependencies. If you have the same USE flags set on all machines, then
> you should not see the need for new dependencies to be brought in on
> any of the non-compile machines.

Hmmm, a simple cat of those three files then, simpler than my idea!

And for the "system" I believe this depends only on the profile chosen, so this 
profile should be the same on each machine, or then I would have to generate the 
list of packages in each pc's "system"...  I don't see why they should be 
different so this should not be a concern.

Thanks!



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-29 23:09 [gentoo-user] Sync'ing and compiling pkgs for multiple PCs Simon
  2008-07-29 22:38 ` Andrey Falko
@ 2008-07-30  8:15 ` Stroller
  2008-07-31  0:59   ` Simon
  1 sibling, 1 reply; 13+ messages in thread
From: Stroller @ 2008-07-30  8:15 UTC (permalink / raw
  To: gentoo-user


On 30 Jul 2008, at 00:09, Simon wrote:
> ...
>   I have 3 computers and they are all setup the same with the  
> exception of the kernel that has different options for the youngest  
> of my pcs.  I'm looking for a simple, easy and efficient way to  
> keep them all up2date.
>
>   I want to avoid 3 --sync to the gentoo servers.  I have tried to  
> setup one of my pc as central server and sync the others on it, but  
> has the main disadvantage that the other PCs are dependant on it  
> (ok, it's not so difficult to change make.conf).

Hi Simon,

Your email is very long, so I'm not sure if I've taken it all in, but  
what I'd suggest is a shared /usr/portage directory (easily done by  
NFS) and distcc.

The shared portage reduces the need to --sync and ensures that all  
computers share access to the binary packages once their built.

distcc is, IMO, a bit more elegant than (for instance) trying to  
manually emerge binary packages for machine A on PC B. You can tell  
it to share the work or just unload it to the most powerful machine.  
There may be concerns about using a binary package if USE flags are  
different between the two machines, but distcc ensures that the  
package is built using those defined in make.conf of the machine on  
which you're running emerge.

http://gentoo-wiki.com/HOWTO_Using_a_shared_portage_via_NFS

http://www.gentoo.org/doc/en/distcc.xml

Stroller.




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-30  8:15 ` Stroller
@ 2008-07-31  0:59   ` Simon
  2008-07-31  9:04     ` Stroller
                       ` (2 more replies)
  0 siblings, 3 replies; 13+ messages in thread
From: Simon @ 2008-07-31  0:59 UTC (permalink / raw
  To: gentoo-user

Hi Stroller, my email was probably as complex as I was confused, but you helped 
a lot...

> Your email is very long, so I'm not sure if I've taken it all in, but 
> what I'd suggest is a shared /usr/portage directory (easily done by NFS) 
> and distcc.

This creates a dependency on the host that contains the portage tree.  It also 
remove some flexibility. For example, taking my laptop away from my 'portage pc' 
would make it impossible for me to modify my current installs while away.  There 
are other consideration and the use of a sync'ed portage tree on each pc vs 
using nfs is a debate that I won't go into now and this part is not much of an 
importance in my problem... compilation is!...

> distcc is, IMO, a bit more elegant than (for instance) trying to 
> manually emerge binary packages for machine A on PC B. You can tell it 
> to share the work or just unload it to the most powerful machine. There 
> may be concerns about using a binary package if USE flags are different 
> between the two machines, but distcc ensures that the package is built 
> using those defined in make.conf of the machine on which you're running 
> emerge.

Yes!  I was actually trying distcc today for the first time and got it working 
from the perspective of my fastest computer, I got some trouble though (see 
below).  What you mentioned about running the `emerge -uDN world` on each 
individual machines + sharing built packages is absolutely awesome. Best of all 
worlds if i could say!

However, when using distcc, I first made a trial with a small package 
'xmahjongg' and got a nice x4 speedup on the overall emerge.  I wanted to try 
with a larger package, 'povray' and stumbled on a linker issue, the issue is 
described below and this is the only obstacle on my way now.  As I fear doing a 
`emerge -e system && emerge -e world` would never complete using distcc...

Doing: `time emerge povray` without distcc yields a functionnal package, while 
when distcc was enabled, I would get lots of undefined references to some 
__pthreads functions.  But I just tried and it seems to work fine, not 
reproducible, so I'll drop my distcc issue and go on with the -e recompilation.

Unless you have some more tips on using distcc on such a large compile?

Thanks again,
   Simon



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-31  0:59   ` Simon
@ 2008-07-31  9:04     ` Stroller
  2008-07-31 11:02       ` Eric Martin
  2008-07-31 14:12       ` Simon
  2008-07-31 16:35     ` Florian Philipp
  2008-08-02 12:11     ` Daniel Iliev
  2 siblings, 2 replies; 13+ messages in thread
From: Stroller @ 2008-07-31  9:04 UTC (permalink / raw
  To: gentoo-user


On 31 Jul 2008, at 01:59, Simon wrote:
> ...
>> Your email is very long, so I'm not sure if I've taken it all in,  
>> but what I'd suggest is a shared /usr/portage directory (easily  
>> done by NFS) and distcc.
>
> This creates a dependency on the host that contains the portage  
> tree.  It also remove some flexibility. For example, taking my  
> laptop away from my 'portage pc' would make it impossible for me to  
> modify my current installs while away.  There are other  
> consideration and the use of a sync'ed portage tree on each pc vs  
> using nfs is a debate that I won't go into now and this part is not  
> much of an importance in my problem... compilation is!...

Yes, your words about "dependency" and "flexibility" are valid, but  
this is also the most straightforward way to sync multiple machines  
at once. If you do need to emerge a package when the laptop is away  
from home then just --sync and it builds a portage tree at the  
"missing mountpoint" (if that makes sense).

You may find it more elegant to make the one machine an rsync server  
for the others.
http://www.gentoo.org/doc/en/rsync.xml

>> distcc is, IMO, a bit more elegant than (for instance) trying to  
>> manually emerge binary packages for machine A on PC B. You can  
>> tell it to share the work or just unload it to the most powerful  
>> machine. There may be concerns about using a binary package if USE  
>> flags are different between the two machines, but distcc ensures  
>> that the package is built using those defined in make.conf of the  
>> machine on which you're running emerge.
>
> Yes!  I was actually trying distcc today for the first time and got  
> it working from the perspective of my fastest computer, I got some  
> trouble though (see below).  What you mentioned about running the  
> `emerge -uDN world` on each individual machines + sharing built  
> packages is absolutely awesome. Best of all worlds if i could say!

Great! I'm glad you're happy with this. You're NFS exporting a sub- 
directory of /usr/portage, then, in order to share the built packages?

> However, when using distcc, I first made a trial with a small  
> package 'xmahjongg' and got a nice x4 speedup on the overall  
> emerge.  I wanted to try with a larger package, 'povray' and  
> stumbled on a linker issue, the issue is described below and this  
> is the only obstacle on my way now.  As I fear doing a `emerge -e  
> system && emerge -e world` would never complete using distcc...
>
> Doing: `time emerge povray` without distcc yields a functionnal  
> package, while when distcc was enabled, I would get lots of  
> undefined references to some __pthreads functions.  But I just  
> tried and it seems to work fine, not reproducible, so I'll drop my  
> distcc issue and go on with the -e recompilation.


I assume that the "undefined references to __pthreads function" are  
errors which stop the compile? Or that they occur when you start the  
app, causing it to crash? Rather than compilation warnings?

I assume compilation errors. My usage is that I can turn distcc off  
for the duration of the compile when I see something like this, and  
not bother investigating it further, but I think the most likely  
cause is that a library is needed for compilation that is not present  
on the distcc server. Portage accepts the compile-time dependency  
because it is filled on the distcc client, the machine on which  
you've run emerge, but when that particular bit is   sent off to the  
distcc server then that machine doesn't have the lib needed.

I would imagine that, assuming the above belief is correct, then the  
workaround would be to `emerge -o` the package on the other machines  
on your LAN (or the fastest machine, if you are using only that to  
emerge) before distcc'ing it. This is slightly inelegant.

If you mostly have the same packages on all machines then hopefully  
you shouldn't encounter this scenario too often, although I'd also  
think that different USE flags could affect it.

I'm also somewhat suspicious of different architectures - you  
wouldn't try compiling for ARM or MIPS on an x86 PC, but I'm not sure  
how compiling on an Athlon for a Pentium 3 or 4 affects things.  
Finally you should make sure all machines are using the same versions  
of gcc and glibc (also binutils? what else?).

Stroller. 
  



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-31  9:04     ` Stroller
@ 2008-07-31 11:02       ` Eric Martin
  2008-07-31 14:12       ` Simon
  1 sibling, 0 replies; 13+ messages in thread
From: Eric Martin @ 2008-07-31 11:02 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 4963 bytes --]

Stroller wrote:
> 
> On 31 Jul 2008, at 01:59, Simon wrote:
>> ...
>>> Your email is very long, so I'm not sure if I've taken it all in, but 
>>> what I'd suggest is a shared /usr/portage directory (easily done by 
>>> NFS) and distcc.
>>
>> This creates a dependency on the host that contains the portage tree.  
>> It also remove some flexibility. For example, taking my laptop away 
>> from my 'portage pc' would make it impossible for me to modify my 
>> current installs while away.  There are other consideration and the 
>> use of a sync'ed portage tree on each pc vs using nfs is a debate that 
>> I won't go into now and this part is not much of an importance in my 
>> problem... compilation is!...
> 
> Yes, your words about "dependency" and "flexibility" are valid, but this 
> is also the most straightforward way to sync multiple machines at once. 
> If you do need to emerge a package when the laptop is away from home 
> then just --sync and it builds a portage tree at the "missing 
> mountpoint" (if that makes sense).
> 
> You may find it more elegant to make the one machine an rsync server for 
> the others.
> http://www.gentoo.org/doc/en/rsync.xml
> 
>>> distcc is, IMO, a bit more elegant than (for instance) trying to 
>>> manually emerge binary packages for machine A on PC B. You can tell 
>>> it to share the work or just unload it to the most powerful machine. 
>>> There may be concerns about using a binary package if USE flags are 
>>> different between the two machines, but distcc ensures that the 
>>> package is built using those defined in make.conf of the machine on 
>>> which you're running emerge.
>>
>> Yes!  I was actually trying distcc today for the first time and got it 
>> working from the perspective of my fastest computer, I got some 
>> trouble though (see below).  What you mentioned about running the 
>> `emerge -uDN world` on each individual machines + sharing built 
>> packages is absolutely awesome. Best of all worlds if i could say!
> 
> Great! I'm glad you're happy with this. You're NFS exporting a 
> sub-directory of /usr/portage, then, in order to share the built packages?
> 
>> However, when using distcc, I first made a trial with a small package 
>> 'xmahjongg' and got a nice x4 speedup on the overall emerge.  I wanted 
>> to try with a larger package, 'povray' and stumbled on a linker issue, 
>> the issue is described below and this is the only obstacle on my way 
>> now.  As I fear doing a `emerge -e system && emerge -e world` would 
>> never complete using distcc...
>>
>> Doing: `time emerge povray` without distcc yields a functionnal 
>> package, while when distcc was enabled, I would get lots of undefined 
>> references to some __pthreads functions.  But I just tried and it 
>> seems to work fine, not reproducible, so I'll drop my distcc issue and 
>> go on with the -e recompilation.
> 
> 
> I assume that the "undefined references to __pthreads function" are 
> errors which stop the compile? Or that they occur when you start the 
> app, causing it to crash? Rather than compilation warnings?
> 
> I assume compilation errors. My usage is that I can turn distcc off for 
> the duration of the compile when I see something like this, and not 
> bother investigating it further, but I think the most likely cause is 
> that a library is needed for compilation that is not present on the 
> distcc server. Portage accepts the compile-time dependency because it is 
> filled on the distcc client, the machine on which you've run emerge, but 
> when that particular bit is   sent off to the distcc server then that 
> machine doesn't have the lib needed.
> 
> I would imagine that, assuming the above belief is correct, then the 
> workaround would be to `emerge -o` the package on the other machines on 
> your LAN (or the fastest machine, if you are using only that to emerge) 
> before distcc'ing it. This is slightly inelegant.
> 
> If you mostly have the same packages on all machines then hopefully you 
> shouldn't encounter this scenario too often, although I'd also think 
> that different USE flags could affect it.
> 
> I'm also somewhat suspicious of different architectures - you wouldn't 
> try compiling for ARM or MIPS on an x86 PC, but I'm not sure how 
> compiling on an Athlon for a Pentium 3 or 4 affects things. Finally you 
> should make sure all machines are using the same versions of gcc and 
> glibc (also binutils? what else?).
> 
> Stroller. 
> 

You can cross compile, I've done it multiple times before When my Athlon 
built packages for my celerons.  Check out the Distcc-Cross Compile 
guide here [1].  Apparently the doc has been updated since I last 
checked as the tools have gotten full rewrites.

1) http://www.gentoo.org/doc/en/cross-compiling-distcc.xml

-- 
Eric Martin
Key fingerprint = D1C4 086E DBB5 C18E 6FDA  B215 6A25 7174 A941 3B9F


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-31  9:04     ` Stroller
  2008-07-31 11:02       ` Eric Martin
@ 2008-07-31 14:12       ` Simon
  2008-07-31 18:15         ` Stroller
  1 sibling, 1 reply; 13+ messages in thread
From: Simon @ 2008-07-31 14:12 UTC (permalink / raw
  To: gentoo-user

> Yes, your words about "dependency" and "flexibility" are valid, but this 
> is also the most straightforward way to sync multiple machines at once. 
> If you do need to emerge a package when the laptop is away from home 
> then just --sync and it builds a portage tree at the "missing 
> mountpoint" (if that makes sense).

I'll consider that, but seriously, I've tried many ways, including rsyncd, to 
sync portage on my computers and I've settled for unison.  Really, I like it for 
its simplicity of use and maintenance and the fact that my host only has the ssh 
port open.  Sync'ing portage is not a problem, i'll explore different ways along 
the way.  Dont try to convince me of other ways, this one is working (and I 
don't mind whatever downsides).

> Great! I'm glad you're happy with this. You're NFS exporting a 
> sub-directory of /usr/portage, then, in order to share the built packages?

As said, no.  I'm using a separate copy on each host, which is sync'ed manually 
between those hosts.  It may sound awful, but it actually syncs my /home 
directory, as well as my /root dir (where I keep important system stuff, like 
dev drivers source).  The sync'ing is not very long and once done, each computer 
is completely independant and contains all my stuff i need.  Which creates 
redundancy.

I had tried NFS before, and failed, but I was not very interested and the first 
few obstacles made me drop it. It was before I tried gentoo and started my own 
personal 'renaissance' and learned so much.  I'll try it again soon, but it's 
not my priority for now.

> I assume compilation errors. My usage is that I can turn distcc off for 
> the duration of the compile when I see something like this, and not 
> bother investigating it further, but I think the most likely cause is 
> that a library is needed for compilation that is not present on the 
> distcc server. Portage accepts the compile-time dependency because it is 
> filled on the distcc client, the machine on which you've run emerge, but 
> when that particular bit is   sent off to the distcc server then that 
> machine doesn't have the lib needed.

Well, this is the whole problem.  You were right about the compilation error. 
And if it *may* happen, then a full emerge -e will take days (considering system 
has 143 pkgs and world has 499 pkgs to build).  Turning it off, emerge the 
package, turn it on, resume emerge once in a while is really against my idea of 
automatism!  I might make a script that checks for binpkgs after emerge returns, 
and if no pkg was created, then disable distcc for that one, re-enable 
automatically...  maybe prepare a report to send to the ebuild maintainer, to 
let him know of the issue and all relevant details.

> I would imagine that, assuming the above belief is correct, then the 
> workaround would be to `emerge -o` the package on the other machines on 
> your LAN (or the fastest machine, if you are using only that to emerge) 
> before distcc'ing it. This is slightly inelegant.

Clearly less and less transparent... i'd say cloudy.  Seriously at this point, 
i'd just drop distcc completely.  But I don't think I'll have to go that far.

> If you mostly have the same packages on all machines then hopefully you 
> shouldn't encounter this scenario too often, although I'd also think 
> that different USE flags could affect it.
> 
> I'm also somewhat suspicious of different architectures - you wouldn't 
> try compiling for ARM or MIPS on an x86 PC, but I'm not sure how 
> compiling on an Athlon for a Pentium 3 or 4 affects things. Finally you 
> should make sure all machines are using the same versions of gcc and 
> glibc (also binutils? what else?).

Same everything on every machine, i'll double check glibc and binutils too.

Thanks Stroller, I'm really starting to see how I can manage my mini IT lab now!
Simon



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-31  0:59   ` Simon
  2008-07-31  9:04     ` Stroller
@ 2008-07-31 16:35     ` Florian Philipp
  2008-08-01  2:44       ` Simon
  2008-08-02 12:11     ` Daniel Iliev
  2 siblings, 1 reply; 13+ messages in thread
From: Florian Philipp @ 2008-07-31 16:35 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

Simon schrieb:
>> Your email is very long, so I'm not sure if I've taken it all in, but 
>> what I'd suggest is a shared /usr/portage directory (easily done by 
>> NFS) and distcc.
> 
> This creates a dependency on the host that contains the portage tree.  
> It also remove some flexibility. For example, taking my laptop away from 
> my 'portage pc' would make it impossible for me to modify my current 
> installs while away.  There are other consideration and the use of a 
> sync'ed portage tree on each pc vs using nfs is a debate that I won't go 
> into now and this part is not much of an importance in my problem... 
> compilation is!...
> 

You might want to take a look at Coda instead of NFS. It supports 
"disconnected" read/write: http://en.wikipedia.org/wiki/Coda_(file_system)

I haven't used it though.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-31 14:12       ` Simon
@ 2008-07-31 18:15         ` Stroller
  2008-08-01  1:34           ` Simon
  0 siblings, 1 reply; 13+ messages in thread
From: Stroller @ 2008-07-31 18:15 UTC (permalink / raw
  To: gentoo-user


On 31 Jul 2008, at 15:12, Simon wrote:

>> Yes, your words about "dependency" and "flexibility" are valid,  
>> but this is also the most straightforward way to sync multiple  
>> machines at once. If you do need to emerge a package when the  
>> laptop is away from home then just --sync and it builds a portage  
>> tree at the "missing mountpoint" (if that makes sense).
>
> I'll consider that, but seriously, I've tried many ways, including  
> rsyncd, to sync portage on my computers and I've settled for  
> unison.  Really, I like it for its simplicity of use and  
> maintenance and the fact that my host only has the ssh port open.   
> Sync'ing portage is not a problem, i'll explore different ways  
> along the way.  Dont try to convince me of other ways, this one is  
> working (and I don't mind whatever downsides).
>
>> Great! I'm glad you're happy with this. You're NFS exporting a sub- 
>> directory of /usr/portage, then, in order to share the built  
>> packages?
>
> As said, no.  I'm using a separate copy on each host, which is  
> sync'ed manually between those hosts.  It may sound awful, but it  
> actually syncs my /home directory, as well as my /root dir (where I  
> keep important system stuff, like dev drivers source).

No, that's fine. Whatever works for you. Basically, as you say, you  
want to keep it as transparent, seamless & automated as possible.  
Whatever does the trick.

I would perhaps do some more investigation & start a new thread about  
distcc problems.

Stroller.




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-31 18:15         ` Stroller
@ 2008-08-01  1:34           ` Simon
  0 siblings, 0 replies; 13+ messages in thread
From: Simon @ 2008-08-01  1:34 UTC (permalink / raw
  To: gentoo-user

> I would perhaps do some more investigation & start a new thread about 
> distcc problems.

Yup, I'll do this major update on friday and report here about any issues 
strictly related to emerge and distcc.  However, no news is good news! =)

Thanks



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-31 16:35     ` Florian Philipp
@ 2008-08-01  2:44       ` Simon
  0 siblings, 0 replies; 13+ messages in thread
From: Simon @ 2008-08-01  2:44 UTC (permalink / raw
  To: gentoo-user

> You might want to take a look at Coda instead of NFS. It supports 
> "disconnected" read/write: http://en.wikipedia.org/wiki/Coda_(file_system)
> 
> I haven't used it though.

Hey thanks!  This seems quite cool!  I've read about it on their website also 
and it would clearly fit all my "requirements" for a network FS.  I believe it 
could even be used to replace completely my current use of unison.

Definitely worth experimenting and if that succeeds, I'll consider migrating to it!

Thanks again!
Simon



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [gentoo-user] Sync'ing and compiling pkgs for multiple PCs
  2008-07-31  0:59   ` Simon
  2008-07-31  9:04     ` Stroller
  2008-07-31 16:35     ` Florian Philipp
@ 2008-08-02 12:11     ` Daniel Iliev
  2 siblings, 0 replies; 13+ messages in thread
From: Daniel Iliev @ 2008-08-02 12:11 UTC (permalink / raw
  To: gentoo-user

On Wed, 30 Jul 2008 20:59:02 -0400
Simon <turner25@gmail.com> wrote:

> 
> > distcc is, IMO, a bit more elegant than (for instance) trying to 
> > manually emerge binary packages for machine A on PC B. You can tell
> > it to share the work or just unload it to the most powerful
> > machine. There may be concerns about using a binary package if USE
> > flags are different between the two machines, but distcc ensures
> > that the package is built using those defined in make.conf of the
> > machine on which you're running emerge.
> 
> Yes!  I was actually trying distcc today for the first time and got
> it working from the perspective of my fastest computer, I got some
> trouble though (see below).  What you mentioned about running the
> `emerge -uDN world` on each individual machines + sharing built
> packages is absolutely awesome. Best of all worlds if i could say!
> 


Perhaps you have tried it already, but I didn't see it mentioned
in this thread, so: for additional speed up of the compilation at the
expense of disk space you may try ccache.

ccache, gentoo.org: http://tinyurl.com/6o2ql

ccache & distcc, gentoo-wiki.com: http://tinyurl.com/85ngj


-- 
Best regards,
Daniel



^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2008-08-02 12:11 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-07-29 23:09 [gentoo-user] Sync'ing and compiling pkgs for multiple PCs Simon
2008-07-29 22:38 ` Andrey Falko
2008-07-29 23:46   ` Simon
2008-07-30  8:15 ` Stroller
2008-07-31  0:59   ` Simon
2008-07-31  9:04     ` Stroller
2008-07-31 11:02       ` Eric Martin
2008-07-31 14:12       ` Simon
2008-07-31 18:15         ` Stroller
2008-08-01  1:34           ` Simon
2008-07-31 16:35     ` Florian Philipp
2008-08-01  2:44       ` Simon
2008-08-02 12:11     ` Daniel Iliev

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox