[gentoo-user] emerge problems

[gentoo-user] emerge problems

[James Stull]

[-- Attachment #1: Type: text/plain, Size: 452 bytes --]

I switched my eselect profile from a generic gentoo system to desktop.
Unfortunately I keep getting this error when I run "emerge -uDNav world" I
tried it with the --skipfirst flag but I continue to have this problem.

 * One or more packages are either masked or have missing dependencies:
 *
 *   =dev-java/java-sdk-docs-1.6.0* pulled in by:
 *     ('installed', '/', 'dev-java/sun-jdk-1.6.0.11', 'nomerge')
 *


Thanks for any help you can provide.

[-- Attachment #2: Type: text/html, Size: 584 bytes --]

Re: [gentoo-user] emerge problems

[pk]

James Stull wrote:
> I switched my eselect profile from a generic gentoo system to desktop.
> Unfortunately I keep getting this error when I run "emerge -uDNav world" I
> tried it with the --skipfirst flag but I continue to have this problem.
> 
>  * One or more packages are either masked or have missing dependencies:
>  *
>  *   =dev-java/java-sdk-docs-1.6.0* pulled in by:
>  *     ('installed', '/', 'dev-java/sun-jdk-1.6.0.11', 'nomerge')
>  *

Hi,

The docs are fetch-restricted; you need to download the documentation
from "http://java.sun.com/javase/6/docs/" and follow the instructions in
the ebuild.

Best regards

Peter K

Re: [gentoo-user] emerge problems

[James Stull]

[-- Attachment #1: Type: text/plain, Size: 786 bytes --]

What is the best way to block this package?


On Sun, Jan 4, 2009 at 6:35 AM, pk <peterk2@coolmail.se> wrote:

> James Stull wrote:
> > I switched my eselect profile from a generic gentoo system to desktop.
> > Unfortunately I keep getting this error when I run "emerge -uDNav world"
> I
> > tried it with the --skipfirst flag but I continue to have this problem.
> >
> >  * One or more packages are either masked or have missing dependencies:
> >  *
> >  *   =dev-java/java-sdk-docs-1.6.0* pulled in by:
> >  *     ('installed', '/', 'dev-java/sun-jdk-1.6.0.11', 'nomerge')
> >  *
>
> Hi,
>
> The docs are fetch-restricted; you need to download the documentation
> from "http://java.sun.com/javase/6/docs/" and follow the instructions in
> the ebuild.
>
> Best regards
>
> Peter K
>
>

[-- Attachment #2: Type: text/html, Size: 1321 bytes --]

[gentoo-user] Re: emerge problems

[Nikos Chantziaras]

James Stull wrote:
> What is the best way to block this package?
> 
> On Sun, Jan 4, 2009 at 6:35 AM, pk <peterk2@coolmail.se> wrote:
> 
>> James Stull wrote:
>>> I switched my eselect profile from a generic gentoo system to desktop.
>>> Unfortunately I keep getting this error when I run "emerge -uDNav world"
>> I
>>> tried it with the --skipfirst flag but I continue to have this problem.
>>>
>>>  * One or more packages are either masked or have missing dependencies:
>>>  *
>>>  *   =dev-java/java-sdk-docs-1.6.0* pulled in by:
>>>  *     ('installed', '/', 'dev-java/sun-jdk-1.6.0.11', 'nomerge')
>>>  *
>> Hi,
>>
>> The docs are fetch-restricted; you need to download the documentation
>> from "http://java.sun.com/javase/6/docs/" and follow the instructions in
>> the ebuild.

Don't use the "doc" USE flag of sun-jdk.  That is, put this in 
/etc/portage/packages.use:

   dev-java/sun-jdk -doc

[gentoo-user] Re: emerge problems

[Nikos Chantziaras]

Nikos Chantziaras wrote:
> /etc/portage/packages.use:
> 
>   dev-java/sun-jdk -doc

Typo.  It's "/etc/portage/package.use".  You need to "emerge -N world" 
after that.

Re: [gentoo-user] Re: emerge problems

[James Stull]

[-- Attachment #1: Type: text/plain, Size: 302 bytes --]

Will do, thanks for your assistance!



On Sun, Jan 4, 2009 at 7:57 PM, Nikos Chantziaras <realnc@arcor.de> wrote:

> Nikos Chantziaras wrote:
>
>> /etc/portage/packages.use:
>>
>>  dev-java/sun-jdk -doc
>>
>
> Typo.  It's "/etc/portage/package.use".  You need to "emerge -N world"
> after that.
>
>
>

[-- Attachment #2: Type: text/html, Size: 779 bytes --]

[gentoo-user] Emerge Problems...

[BRM]

A little while back my server ran out of hard disk space (due to a failed hard 
drive) and as a result my local portage mirror got destroyed.
Well, I fixed there server - initially by just grabbing a new copy of portage 
like a new install since it was just completely hosed, and the server is back up 
and working. However, now my desktop and laptop are both having problems. They 
sync just fine against the server, but I get a series of errors about not having 
various ebuilds in the manifest files - so many that I can't emerge anything 
(even portage).

Right now, my laptop is basically hosed - KDE/X won't work on login due to some 
errors. My desktop at least logs in to the KDE/X. However, on both systems I am 
having the manifest problem, and I can't edit files either since vim is screwed 
up due to a change in perl - and I can't run perl-clearner due to the emerge 
problem.

I know both systems can be restored to being fully functional and up-to-date. 
The question is - how do I get there?

I ran across some emails in the list archive on a similar issue - though that 
was only for 1 ebuild - and it was straight forward enough to fix by just 
rebuilding the manifest though 'ebuild' or something. I ran across another 
e-mail suggesting to just resync, and well - I tried that but it didn't work.

So my question is - is there a way to automatically fix all these manifest 
things without having to track down each one by hand and run the 'ebuild' thing 
on each one individually? I'm completely out of ideas, and I'd really like to 
get these systems back to full functionality.

TIA,

Ben

[gentoo-user] Re: Emerge Problems...

[Francesco Talamona]

On Sunday 30 January 2011, BRM wrote:
> A little while back my server ran out of hard disk space (due to a
> failed hard drive) and as a result my local portage mirror got
> destroyed. Well, I fixed there server - initially by just grabbing a
> new copy of portage like a new install since it was just completely
> hosed, and the server is back up and working. However, now my
> desktop and laptop are both having problems. They sync just fine
> against the server, but I get a series of errors about not having
> various ebuilds in the manifest files - so many that I can't emerge
> anything (even portage).

It seems your three systems share a broken portage tree, try with the 
latest portage snapshot, for example from 
http://distro.ibiblio.org/pub/linux/distributions/gentoo/snapshots/

You can also skip the sync and put it directly on the clients to see if 
the rsync service on server is broken... 

Once you stabilize the root cause, it's time to focus on the other 
issues (for example run a non-X runlevel on the laptop to fix the login 
issue, use nano until vim is ok, and so on).

HTH
	Francesco

-- 
Linux Version 2.6.36-gentoo-r6, Compiled #2 SMP PREEMPT Mon Jan 3 
11:54:58 CET 2011
Two 1GHz AMD Athlon 64 Processors, 4GB RAM, 4021.84 Bogomips Total
aemaeth

Re: [gentoo-user] Emerge Problems...

[Nils Holland]

On 20:12 Sat 29 Jan     , BRM wrote:
> A little while back my server ran out of hard disk space (due to a failed hard 
> drive) and as a result my local portage mirror got destroyed.
> Well, I fixed there server - initially by just grabbing a new copy of portage 
> like a new install since it was just completely hosed, and the server is back up 
> and working. However, now my desktop and laptop are both having problems. They 
> sync just fine against the server, but I get a series of errors about not having 
> various ebuilds in the manifest files - so many that I can't emerge anything 
> (even portage).

I believe you will already have checked this, but anyways:

I once upon a time experienced a similar issue, which was caused by the fact that for some reason, I was only syncing new / modfied files from the source to my local portage tree, and not deleting no longer existent (on the source) files from the local tree. This resulted in emerge complaining about various ebuilds not being found.

I was kind of shocked at first, then found my error, and on "properly" (including deletes) syncing with my portage source everything immediately started working fine again on the local (destination) machine.

But again, I believe it's highly unprobable that this is your problem, because if you synced "correctly" before your server had to be re-setup, I would believe that you're doing it correctly now as well, at least I can't see what should have changed concering the sync due to the act of replacing the server...

Greetings,
Nils


-- 
Nils Holland * Ti Systems, Wunsorf-Luthe (Germany)
Powered by GNU/Linux since 1998

Re: [gentoo-user] Emerge Problems...

[BRM]

----- Original Message ----

> From: Nils Holland <nhg@tisys.org>
> On 20:12 Sat 29 Jan     , BRM wrote:
> > A little while back my  server ran out of hard disk space (due to a failed 
>hard 
>
> > drive) and as a  result my local portage mirror got destroyed.
> > Well, I fixed there server  - initially by just grabbing a new copy of 
>portage 
>
> > like a new install  since it was just completely hosed, and the server is 
>back up 
>
> > and  working. However, now my desktop and laptop are both having problems. 
>They 
>
> > sync just fine against the server, but I get a series of errors about  not 
>having 
>
> > various ebuilds in the manifest files - so many that I can't  emerge anything 
>
> > (even portage).
> 
> I believe you will already have  checked this, but anyways:
> 
> I once upon a time experienced a similar  issue, which was caused by the fact 
>that for some reason, I was only syncing new  / modfied files from the source to 
>my local portage tree, and not deleting no  longer existent (on the source) 
>files from the local tree. This resulted in  emerge complaining about various 
>ebuilds not being found.
> 
> I was kind of  shocked at first, then found my error, and on "properly" 
>(including deletes)  syncing with my portage source everything immediately 
>started working fine again  on the local (destination) machine.
> 
> But again, I believe it's highly  unprobable that this is your problem, because 
>if you synced "correctly" before  your server had to be re-setup, I would 
>believe that you're doing it correctly  now as well, at least I can't see what 
>should have changed concering the sync  due to the act of replacing the  
>server...
> 

May be I didn't get the server back up right? Not sure.
Any how...the primary issue was resolved once I "deleted" the server's portage 
mirror and than ran rsync again to grab a fresh copy.
I'm pretty sure it would have to be how I rsync'd the mirror since I lost my 
mirroring script the old hard drive died.
I just wrote a new script last night, but I'm still not sure that all of the 
parameters are correct - I'll check into that more this evening.
Once I get it right, I'll restore it do doing the daily mirror syncs again.

Now I just have to get past all the issues coming up in the updates and 
rebuilds, but that was to be expected.

Thanks!

Ben

[gentoo-user] Re: Emerge Problems...

[Francesco Talamona]

On Monday 31 January 2011, BRM wrote:
> I just wrote a new script last night, but I'm still not sure that all
> of the  parameters are correct

Why not something proven and reliable like "emerge --sync"?

Ciao
	Francesco

-- 
Linux Version 2.6.36-gentoo-r6, Compiled #2 SMP PREEMPT Mon Jan 3 
11:54:58 CET 2011
Two 1GHz AMD Athlon 64 Processors, 4GB RAM, 4021.84 Bogomips Total
aemaeth

Re: [gentoo-user] Re: Emerge Problems...

[BRM]

----- Original Message ----

> From: Francesco Talamona <francesco.talamona@know.eu>
> On Monday 31 January 2011, BRM wrote:
> > I just wrote a new script last  night, but I'm still not sure that all
> > of the  parameters are  correct
> 
> Why not something proven and reliable like "emerge  --sync"?
> 

"emerge --sync" works fine for your _normal_ portage tree.
But if you are running a mirror on a gentoo system that also needs its own copy 
of portage, then you really need to have two portage trees on the system.
One portage tree is hosted by rsync for all - it can be synch'd at will with the 
official portage trees.
The second portage tree is the system's portage tree, and is only sync'd when 
you update it - just like any other gentoo system.

Why?

I originally ran the server with rsync hosting its portage tree, with daily 
synchronizations. However, when I forgot and let the server fall behind a little 
in updates, it became quickly clear that it needed its own separate copy of 
portage so I can install software without synchronizing portage - or rather, 
install software without having to update the whole system, etc.

Now, may be there are options for "emerge --sync" that I'm not aware of to 
handle just this case - but it works very well, and I ran it for quite a while. 
Sadly, I did not have that script backed up or anything; so I will have to 
recreate it again.

Ben

Re: [gentoo-user] Re: Emerge Problems...

[Peter Humphrey]

[-- Attachment #1: Type: text/plain, Size: 675 bytes --]

On Monday 31 January 2011 22:26:20 BRM wrote:

> "emerge --sync" works fine for your _normal_ portage tree.
> But if you are running a mirror on a gentoo system that also needs
> its own copy of portage, then you really need to have two portage
> trees on the system. One portage tree is hosted by rsync for all -
> it can be synch'd at will with the official portage trees.
> The second portage tree is the system's portage tree, and is only
> sync'd when you update it - just like any other gentoo system.

I don't understand any of this. Why should any two systems require 
different versions of the portage tree?

-- 
Rgds
Peter.          Linux Counter 5290, 1994-04-23.

[-- Attachment #2: Type: text/html, Size: 3218 bytes --]

Re: [gentoo-user] Re: Emerge Problems...

[Nils Holland]

On 21:35 Mon 31 Jan     , Francesco Talamona wrote:
> On Monday 31 January 2011, BRM wrote:
> > I just wrote a new script last night, but I'm still not sure that all
> > of the  parameters are correct
> 
> Why not something proven and reliable like "emerge --sync"?

In fact, what I always do is sync one of my machines with an official
Gentoo mirror via "emerge --sync", and then I just use rsync to
distribute the updated tree to all my other local machines as in:

rsync --delete -trmv /usr/portage/ <user>@<dest_host>:/usr/portage

One might want to ask rsync to exclude the distfiles directory,
but I always include it as it oftentimes saves me the download of a
file I've already downloaded during an emerge on another machine.

In any case, locally updating my tree via rsync has always worked fine
for me. Leaving the "--delete" option to rsync out, however,
immediately leads to problems, with various ebuild-related error
messages on subsequent "emerge"s. I can imagine that the OP did, in
fact, update his tree in such an inconsistent manner, but that can
certainly be fixed, with the surest way being a "emerge --sync" using
an official mirror.

Greetings,
Nils


-- 
Nils Holland * Ti Systems, Wunsorf-Luthe (Germany)
Powered by GNU/Linux since 1998

Re: [gentoo-user] Re: Emerge Problems...

[Dale]

Nils Holland wrote:
> On 21:35 Mon 31 Jan     , Francesco Talamona wrote:
>    
>> On Monday 31 January 2011, BRM wrote:
>>      
>>> I just wrote a new script last night, but I'm still not sure that all
>>> of the  parameters are correct
>>>        
>> Why not something proven and reliable like "emerge --sync"?
>>      
> In fact, what I always do is sync one of my machines with an official
> Gentoo mirror via "emerge --sync", and then I just use rsync to
> distribute the updated tree to all my other local machines as in:
>
> rsync --delete -trmv /usr/portage/<user>@<dest_host>:/usr/portage
>
> One might want to ask rsync to exclude the distfiles directory,
> but I always include it as it oftentimes saves me the download of a
> file I've already downloaded during an emerge on another machine.
>
> In any case, locally updating my tree via rsync has always worked fine
> for me. Leaving the "--delete" option to rsync out, however,
> immediately leads to problems, with various ebuild-related error
> messages on subsequent "emerge"s. I can imagine that the OP did, in
> fact, update his tree in such an inconsistent manner, but that can
> certainly be fixed, with the surest way being a "emerge --sync" using
> an official mirror.
>
> Greetings,
> Nils
>    

Maybe I am missing something but I have two machines here.  I sync to 
the Gentoo servers with the main rig and then sync the second rig from 
the main rig.  All you have to do is start the rsync service and set the 
IP address in the SYNC line in make.conf on the second rig.  This is my 
rsyncd.conf on the main rig:

# Simple example for enabling your own local rsync server
[gentoo-portage]
path = /usr/portage
comment = Gentoo Portage tree
exclude = /distfiles /packages

If you want to include distfiles, just remove it from the exclude line.  
For my distfiles, I run http-replicator to fetch those.  It works pretty 
well.

Dale

:-)  :-)

Re: [gentoo-user] Re: Emerge Problems...

[BRM]

----- Original Message ----

> From: Dale <rdalek1967@gmail.com>
> Nils Holland wrote:
> > On 21:35 Mon 31 Jan     , Francesco  Talamona wrote:
> >> On Monday 31 January 2011, BRM  wrote:
> >>> I just wrote a new  script last night, but I'm still not sure that all
> >>> of the   parameters are correct
> >>>        
> >>  Why not something proven and reliable like "emerge --sync"?
> >>       
> > In fact, what I always do is sync one of my machines with  an official
> > Gentoo mirror via "emerge --sync", and then I just use rsync  to
> > distribute the updated tree to all my other local machines as  in:
> > 
> > rsync --delete -trmv  /usr/portage/<user>@<dest_host>:/usr/portage
> > 
> > One  might want to ask rsync to exclude the distfiles directory,
> > but I always  include it as it oftentimes saves me the download of a
> > file I've already  downloaded during an emerge on another machine.
> > 
> > In any case,  locally updating my tree via rsync has always worked fine
> > for me.  Leaving the "--delete" option to rsync out, however,
> > immediately leads  to problems, with various ebuild-related error
> > messages on subsequent  "emerge"s. I can imagine that the OP did, in
> > fact, update his tree in  such an inconsistent manner, but that can
> > certainly be fixed, with the  surest way being a "emerge --sync" using
> > an official mirror.
> > 

Definitely missed the delete option on the new script.

> Maybe I am missing  something but I have two machines here.  I sync to the 
>Gentoo servers with  the main rig and then sync the second rig from the main 
>rig.  All you have  to do is start the rsync service and set the IP address in 
>the SYNC line in  make.conf on the second rig.  This is my rsyncd.conf on the 
>main  rig:
> 
> # Simple example for enabling your own local rsync  server
> [gentoo-portage]
> path = /usr/portage
> comment = Gentoo Portage  tree
> exclude = /distfiles /packages
> 
> If you want to include distfiles,  just remove it from the exclude line.  For 
>my distfiles, I run  http-replicator to fetch those.  It works pretty  well.
> 

If the machine you are hosting portage on (via rsync) is fast enough to complete 
all updates within the update cycle (e.g. sync'ing 1 time a day, so it has 
23:59:59 to complete all builds) then it is likely not a problem to do as that.

If the machine is not fast enough - mine is a PII 233 w/160 MB RAM, takes a 
while do to updates - then you really have to separate out what you are hosting 
from what you are using. Otherwise you end up in the situation that you have 
started one system update (or software install), have a build failure for 
whatever reason, and then can't complete the same one due to changes in the 
local copy of portage.

So, even if your system fell into the first situation - where it is fast enough 
- then I would still recommend doing the little extra to run as the second 
situation. It's just far easier to maintain. I'm actually surprised the Gentoo 
Mirror documentation doesn't recommend doing this to start with, but then again 
- the machine they recommend are magnitudes faster than what I'm running so it's 
not likely an issue. (Either that or everyone figures it out on their own and 
then just doesn't say anything.)

Why?

The local portage copy is always up-to-date, or reasonably so. No - I don't sync 
every 1/2 hour (like the official mirrors do), but I could force it to sync when 
I need to if that was an issue; typically once a day is sufficient and that's 
run by a cron job. But I also keep my server system relatively stable - I don't 
install a lot of software on it, and I don't necessarily update it frequently. 
So now I can update my laptop and desktop as well without having to first update 
the server itself since the rsync hosted portage is independent of the server.

Ben

Re: [gentoo-user] Re: Emerge Problems...

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1211 bytes --]

On Tue, 1 Feb 2011 05:48:32 -0800 (PST), BRM wrote:

> If the machine is not fast enough - mine is a PII 233 w/160 MB RAM,
> takes a while do to updates - then you really have to separate out what
> you are hosting from what you are using. Otherwise you end up in the
> situation that you have started one system update (or software
> install), have a build failure for whatever reason, and then can't
> complete the same one due to changes in the local copy of portage.

You can still use emerge -sync instead of a home brewed script. In make
conf, set SYNC to localhost, then in your cron job, do

SYNC="some gentoo rsync mirror" emerge --sync

> So, even if your system fell into the first situation - where it is
> fast enough 
> - then I would still recommend doing the little extra to run as the
> second situation. It's just far easier to maintain.

I've been using a single portage tree to serve a LAN and for use by the
host for years with no hint of any of the problems you suggest. I just
make sure the cron job on the server syncs earlier than the rest of the
LAN and everything is up to date.


-- 
Neil Bothwick

Politics: Poli (many) - tics (blood sucking parasites)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Re: Emerge Problems...

[Dale]

Neil Bothwick wrote:
> On Tue, 1 Feb 2011 05:48:32 -0800 (PST), BRM wrote:
>
>    
>> If the machine is not fast enough - mine is a PII 233 w/160 MB RAM,
>> takes a while do to updates - then you really have to separate out what
>> you are hosting from what you are using. Otherwise you end up in the
>> situation that you have started one system update (or software
>> install), have a build failure for whatever reason, and then can't
>> complete the same one due to changes in the local copy of portage.
>>      
> You can still use emerge -sync instead of a home brewed script. In make
> conf, set SYNC to localhost, then in your cron job, do
>
> SYNC="some gentoo rsync mirror" emerge --sync
>
>    
>> So, even if your system fell into the first situation - where it is
>> fast enough
>> - then I would still recommend doing the little extra to run as the
>> second situation. It's just far easier to maintain.
>>      
> I've been using a single portage tree to serve a LAN and for use by the
> host for years with no hint of any of the problems you suggest. I just
> make sure the cron job on the server syncs earlier than the rest of the
> LAN and everything is up to date.
>
>    

I used to have four computers a good while back.  Back then, I synced my 
main rig then synced the others off it.  This was several years ago.  I 
don't use a cron job or anything to do this, just some old fashioned 
typing.  I don't recall ever having trouble with it syncing to my main 
rig.  Did I mention it was a very old Compaq 200MHz CPU machine with a 
whopping 128MBs of ram?  Thing looks like a filing cabinet.

To me, it seems the OP is making something complicated when it is just 
not needed.  If you want to use cron jobs, set the main rig to sync a 
hour before the others would be set to sync against it.  If the rig that 
syncs to Gentoo servers is to slow, set them two hours apart.  From my 
understanding, you get the same tree all the way around.

Giving some more thought, I once put /usr/portage on nfs.  I sync once 
and all the systems used the same copy of the tree.  The other way 
worked out to be easier tho.  I seem to recall the need for running 
emerge --metadata too.  That took a while on the old Compaq.  lol

Dale

:-)  :-)

Re: [gentoo-user] Re: Emerge Problems...

[Joshua Murphy]

On Tue, Feb 1, 2011 at 12:20 PM, Dale <rdalek1967@gmail.com> wrote:
> Neil Bothwick wrote:
>>
>> On Tue, 1 Feb 2011 05:48:32 -0800 (PST), BRM wrote:
>>
>>
>>>
>>> If the machine is not fast enough - mine is a PII 233 w/160 MB RAM,
>>> takes a while do to updates - then you really have to separate out what
>>> you are hosting from what you are using. Otherwise you end up in the
>>> situation that you have started one system update (or software
>>> install), have a build failure for whatever reason, and then can't
>>> complete the same one due to changes in the local copy of portage.
>>>
>>
>> You can still use emerge -sync instead of a home brewed script. In make
>> conf, set SYNC to localhost, then in your cron job, do
>>
>> SYNC="some gentoo rsync mirror" emerge --sync
>>
>>
>>>
>>> So, even if your system fell into the first situation - where it is
>>> fast enough
>>> - then I would still recommend doing the little extra to run as the
>>> second situation. It's just far easier to maintain.
>>>
>>
>> I've been using a single portage tree to serve a LAN and for use by the
>> host for years with no hint of any of the problems you suggest. I just
>> make sure the cron job on the server syncs earlier than the rest of the
>> LAN and everything is up to date.
>>
>>
>
> I used to have four computers a good while back.  Back then, I synced my
> main rig then synced the others off it.  This was several years ago.  I
> don't use a cron job or anything to do this, just some old fashioned typing.
>  I don't recall ever having trouble with it syncing to my main rig.  Did I
> mention it was a very old Compaq 200MHz CPU machine with a whopping 128MBs
> of ram?  Thing looks like a filing cabinet.
>
> To me, it seems the OP is making something complicated when it is just not
> needed.  If you want to use cron jobs, set the main rig to sync a hour
> before the others would be set to sync against it.  If the rig that syncs to
> Gentoo servers is to slow, set them two hours apart.  From my understanding,
> you get the same tree all the way around.
>
> Giving some more thought, I once put /usr/portage on nfs.  I sync once and
> all the systems used the same copy of the tree.  The other way worked out to
> be easier tho.  I seem to recall the need for running emerge --metadata too.
>  That took a while on the old Compaq.  lol
>
> Dale
>
> :-)  :-)

The trick I've been using for... a couple years now, across various
machines (no cron involved), is syncing one box that shares portage
*and* my distfiles on nfs, portage R/O, distfiles R/W, then when it's
done syncing and starts its own metadata update, hop across all the
others and do an emerge --metadata. Once each one finishes, run
through their individual updates. Because distfiles is shared, and
portage's distfile locking is done right... I download each tarball of
sources exactly once, even when 5-6 machines might share the same one.
I've been quite pleased by that... even more handy is the shared git
pull of wine that I build against on 3 different boxes (I tend to
stagger those rebuilds though, haven't risked finding out if that
would clash).

-- 
Poison [BLX]
Joshua M. Murphy

Re: [gentoo-user] Re: Emerge Problems...

[Nils Holland]

On 15:41 Tue 01 Feb     , Joshua Murphy wrote:
> 
> The trick I've been using for... a couple years now, across various
> machines (no cron involved), is syncing one box that shares portage
> *and* my distfiles on nfs, portage R/O, distfiles R/W, then when it's
> done syncing and starts its own metadata update, hop across all the
> others and do an emerge --metadata.

From the emerge man page: "In versions of portage >=2.1.5 the
--metadata action is totally unnecessary unless the user has enabled
FEATURES="metadata-trasfer" in make.conf(5)."

Could mean that you can skip this --metadata step on your other
machines?

Greetings,
Nils


-- 
Nils Holland * Ti Systems, Wunsorf-Luthe (Germany)
Powered by GNU/Linux since 1998

Re: [gentoo-user] Re: Emerge Problems...

[BRM]

----- Original Message ----
> From: Dale <rdalek1967@gmail.com>
> To: gentoo-user@lists.gentoo.org
> Sent: Tue, February 1, 2011 12:20:56 PM
> Subject: Re: [gentoo-user] Re: Emerge Problems...
> 
> Neil Bothwick wrote:
> > On Tue, 1 Feb 2011 05:48:32 -0800 (PST), BRM  wrote:
> > 
> >    
> >> If the machine is not fast  enough - mine is a PII 233 w/160 MB RAM,
> >> takes a while do to updates  - then you really have to separate out what
> >> you are hosting from  what you are using. Otherwise you end up in the
> >> situation that you  have started one system update (or software
> >> install), have a build  failure for whatever reason, and then can't
> >> complete the same one  due to changes in the local copy of portage.
> >>      
> > You can still use emerge -sync instead of a home brewed script. In  make
> > conf, set SYNC to localhost, then in your cron job, do
> > 
> > SYNC="some gentoo rsync mirror" emerge --sync
> > 
> >     
> >> So, even if your system fell into the first situation -  where it is
> >> fast enough
> >> - then I would still recommend  doing the little extra to run as the
> >> second situation. It's just far  easier to maintain.
> >>      
> > I've been using a  single portage tree to serve a LAN and for use by the
> > host for years  with no hint of any of the problems you suggest. I just
> > make sure the  cron job on the server syncs earlier than the rest of the
> > LAN and  everything is up to date.
> > 
> >    
> 
> I used to have  four computers a good while back.  Back then, I synced my main 
>rig then  synced the others off it.  This was several years ago.  I don't use a  
>cron job or anything to do this, just some old fashioned typing.  I don't  
>recall ever having trouble with it syncing to my main rig.  Did I mention  it 
>was a very old Compaq 200MHz CPU machine with a whopping 128MBs of ram?   Thing 
>looks like a filing cabinet.
> 
> To me, it seems the OP is making  something complicated when it is just not 
>needed.  If you want to use cron  jobs, set the main rig to sync a hour before 
>the others would be set to sync  against it.  If the rig that syncs to Gentoo 
>servers is to slow, set them  two hours apart.  From my understanding, you get 
>the same tree all the way  around.
> 
> Giving some more thought, I once put /usr/portage on nfs.  I  sync once and all 
>the systems used the same copy of the tree.  The other  way worked out to be 
>easier tho.  I seem to recall the need for running  emerge --metadata too.  That 
>took a while on the old Compaq.   lol
> 

And you're doing a typically manual process for updating all the systems - 
update your server first, then any rsync clients. Fine & dandy if that is your 
process - but it's not mine. I may update my laptop twice as often as the other 
two, especially if I want to play with some software or try something out, or 
fix a bug, or get a later version of KDE. The server gets updated may be once a 
month, while the laptop is either once a month or at whim when I want something 
that just came out.

It's not harder to do it this way, just a different method. The original rsync 
script worked perfectly fine; the broken update I did when I lost it is what 
started this whole thread.

As the old saying goes - Different Strokes for Different Folks.

Ben

Re: [gentoo-user] Re: Emerge Problems...

[Dale]

BRM wrote:
> And you're doing a typically manual process for updating all the systems -
> update your server first, then any rsync clients. Fine&  dandy if that is your
> process - but it's not mine. I may update my laptop twice as often as the other
> two, especially if I want to play with some software or try something out, or
> fix a bug, or get a later version of KDE. The server gets updated may be once a
> month, while the laptop is either once a month or at whim when I want something
> that just came out.
>
> It's not harder to do it this way, just a different method. The original rsync
> script worked perfectly fine; the broken update I did when I lost it is what
> started this whole thread.
>
> As the old saying goes - Different Strokes for Different Folks.
>
> Ben
>
>    

Again, maybe I am missing something but it doesn't really matter how 
often you update.  Some people sync their main server and test packages, 
upgrade some stuff figure out a few workarounds then later on sync the 
other machines against the main server.  The portage tree may be days 
old on the main server by that time but at least you know what you are 
up against if you are updating a LOT of systems.

As you say tho, different strokes.

Dale

:-)  :-)

Re: [gentoo-user] Re: Emerge Problems...

[Peter Humphrey]

[-- Attachment #1: Type: text/plain, Size: 1035 bytes --]

On Tuesday 01 February 2011 20:43:43 BRM wrote:

> And you're doing a typically manual process for updating all the
> systems - update your server first, then any rsync clients. Fine &
> dandy if that is your process - but it's not mine. I may update my
> laptop twice as often as the other two, especially if I want to play
> with some software or try something out, or fix a bug, or get a
> later version of KDE. The server gets updated may be once a month,
> while the laptop is either once a month or at whim when I want
> something that just came out.
> 
> It's not harder to do it this way, just a different method. The
> original rsync script worked perfectly fine; the broken update I did
> when I lost it is what started this whole thread.

What's wrong with keeping your server's portage cache up to date? You 
don't have to update the server from it if you don't want to, but if the 
cache is out of date it isn't being much of a server.

I recommend Occam's Razor.

-- 
Rgds
Peter.          Linux Counter 5290, 1994-04-23.

[-- Attachment #2: Type: text/html, Size: 4598 bytes --]

Re: [gentoo-user] Re: Emerge Problems...

[BRM]

>From: Peter Humphrey <peter@humphrey.ukfsn.org>
>On Tuesday 01 February 2011 20:43:43 BRM wrote:
>> And you're doing a typically manual process for updating all the
>> systems - update your server first, then any rsync clients. Fine &
>> dandy if that is your process - but it's not mine. I may update my
>> laptop twice as often as the other two, especially if I want to play
>> with some software or try something out, or fix a bug, or get a
>> later version of KDE. The server gets updated may be once a month,
>> while the laptop is either once a month or at whim when I want
>> something that just came out.
>> 
>> It's not harder to do it this way, just a different method. The
>> original rsync script worked perfectly fine; the broken update I did
>> when I lost it is what started this whole thread.
>What's wrong with keeping your server's portage cache up to date? You don't have 
>
>to update the server from it if you don't want to, but if the cache is out of 
>date it isn't being much of a server.
>I recommend Occam's Razor.
>-- 

Here's the problem with the Server's /usr/portage being hosted by rsync:

- Server sync's its portage against gentoo mirrors (emerge --sync)
- Update Server (emerge world -vuDN)
- Client sync's its portage against server portage mirror (emerge --sync)
- Update Client (emerge world -vudN)

So if you are manually updating the server, then no problem - you control the 
timing.

Now all that seems to work fine until you introduce the automatic updates of the 
server's portage, e.g. via cron.
Suppose the Server Update doesn't complete due to a build error. If the server 
automatically updated its portage during the build time then when you go to redo 
the build you may end up with another set of updates to push in, meanwhile you 
haven't finished the last round. Sure, the clients will still update just fine - 
it's not a problem for _them_, it's a problem for the server.

So, Occam's Razor - store the rsync hosted portage mirror separately from the 
server's /usr/portage copy, and sync the server against the local rsync just 
like all the clients.
The rsync hosted mirror can now be updated at will without any repercussions to 
any install, and the server works just like any of the clients; so now you end 
up with:

- sync server portage mirror against gentoo mirrors at scheduled intervals, e.g. 
every day at midnight
- Server sync's its portage against server portage mirror (emerge --sync)
- Update Server (emerge world -vuDN)
- Client sync's its portage against server portage mirror (emerge --sync)
- Update Client (emerge world -vudN)

The server is now completely 100% independent of the portage it is hosting for 
everyone else on the internal network, and you can get through a full update - 
resolving all issues, etc. - before any re-syncing.

So then the question becomes why run the night cron to update the server's 
portage mirror?
B/c I am not updating or installing software on my server as frequently as my 
other systems; so it doesn't need to be in sync itself as frequently.

Ben

Re: [gentoo-user] Re: Emerge Problems...

[Nils Holland]

On 18:13 Mon 31 Jan     , Dale wrote:
> Nils Holland wrote:
> >      
> > In fact, what I always do is sync one of my machines with an official
> > Gentoo mirror via "emerge --sync", and then I just use rsync to
> > distribute the updated tree to all my other local machines as in:
> >
> > rsync --delete -trmv /usr/portage/ <user>@<dest_host>:/usr/portage
> >
> > One might want to ask rsync to exclude the distfiles directory,
> > but I always include it as it oftentimes saves me the download of a
> > file I've already downloaded during an emerge on another machine.
> 
> Maybe I am missing something but I have two machines here.  I sync to 
> the Gentoo servers with the main rig and then sync the second rig from 
> the main rig.  All you have to do is start the rsync service and set the 
> IP address in the SYNC line in make.conf on the second rig.  This is my 
> rsyncd.conf on the main rig:
> [...]

That actually makes sense, it would mean that rsyncd would have to be
running and appropriately configured on the local "master" machine,
and then it would provide the advantage of being able to sync all
other local machines with the local master via a standard "emerge
--sync" instead of a relatively long rsync command. Indeed, I guess
I'll start doing this here as well.

However, I have a server hosted at some hosting company as well, and I
prefer not to sync it with an official Gentoo mirror, but with my
local portage tree, in order to be sure that I have the exact same
version of the portage tree on the server that I also use locally. For
that case, NAT would prevent my server from contacting an rsync daemon
on a local machine, so I'm actually using a locally invoked rsync to
"shove" the tree to the server (vs. having the server "fetch" it). I
guess that can't easily be changed, but it's not a problem anyway, as
my current mode of operation works well.

Greetings,
Nils


-- 
Nils Holland * Ti Systems, Wunsorf-Luthe (Germany)
Powered by GNU/Linux since 1998