[gentoo-dev] package.mask or package.mask.d

[gentoo-dev] package.mask or package.mask.d

[William Hubbs]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, Aug 22, 2009 at 08:39:47PM +0100, Ciaran McCreesh wrote:
> We also need to consider whether people even want it done exactly the
> way Portage does it now. Some developers have expressed a preference
> for a package.mask.d of some kind instead.

I saw that, and I'm not sure why they suggested changing the directory
from package.mask to package.mask.d, since all you would need to do is
rename the file package.mask to something like package.mask/oldmasks and
the masks in it would be preserved until you put them in different files
in the package.mask directory and removed them from oldmasks, ultimately
deleting oldmasks.


- -- 
William Hubbs
gentoo accessibility team lead
williamh@gentoo.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkqQWqYACgkQblQW9DDEZTiKxQCfejjxnM/8EmhXglK6bpnzCxIG
emcAn3CFgDOJ27wkNWo46DZh2p/N5J74
=v+g+
-----END PGP SIGNATURE-----

[gentoo-dev] Re: package.mask or package.mask.d

[Duncan]

William Hubbs posted on Sat, 22 Aug 2009 15:52:54 -0500 as excerpted:

> On Sat, Aug 22, 2009 at 08:39:47PM +0100, Ciaran McCreesh wrote:
>> We also need to consider whether people even want it done exactly the
>> way Portage does it now. Some developers have expressed a preference
>> for a package.mask.d of some kind instead.
> 
> I saw that, and I'm not sure why they suggested changing the directory
> from package.mask to package.mask.d, since all you would need to do is
> rename the file package.mask to something like package.mask/oldmasks and
> the masks in it would be preserved until you put them in different files
> in the package.mask directory and removed them from oldmasks, ultimately
> deleting oldmasks.

The (one) idea is to keep package.mask as a file for old package managers 
that don't know about package.* as a directory yet.  Moving package.mask 
(the file) to package.mask/oldmasks wouldn't accomplish that.   The 
scenario people are worried about, is where (for example) someone has 
gone away for their year's service (or two, or whatever) that some 
nations have, and comes back and tries to upgrade, only to find the tree 
is so changed that as soon as they sync, their old package manager is 
broken, to the point they can't use use it to upgrade to a new version, 
in ordered to be able to upgrade everything else!

Actually splitting the current single file into multiple files isn't 
really a problem, and would probably be done at the same time package.mask
(.d) is made a directory, all in the same commit, so it's nothing to 
worry about. 

That said, in this particular instance, I don't believe that should be a 
big problem.  Why?  Because portage has supported it for years already, 
and anyone using a PM that doesn't currently support it has by definition 
already demonstrated they are reasonably active about their Gentoo based 
system management choices, and thus isn't likely to let a system go 
unsynced and the PM unupdated for greater than say 90 days anyway.  Thus, 
when the policy is approved by council, stick a 90-180 day hold on 
changing old profiles in the tree, and be done with it.

So ancient PMs shouldn't be a problem either way, here.  Which leaves 
only a simple bikeshedding issue, whether people prefer keeping the names 
we have, or switching to the *.d forms in keeping with the pattern used 
for all those /etc/*.d directories, of which a quick ls here demonstrated 
there's about double the number I expected, having never actually counted 
them before.  While I'd vote for sticking with what we have, that /is/ 
just bikeshedding, and I REALLY want to just get on with it, regardless 
of what it's named, as the feature really is quite useful.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: package.mask or package.mask.d

[Dale]

Duncan wrote:
> William Hubbs posted on Sat, 22 Aug 2009 15:52:54 -0500 as excerpted:
>
>   
>> On Sat, Aug 22, 2009 at 08:39:47PM +0100, Ciaran McCreesh wrote:
>>     
>>> We also need to consider whether people even want it done exactly the
>>> way Portage does it now. Some developers have expressed a preference
>>> for a package.mask.d of some kind instead.
>>>       
>> I saw that, and I'm not sure why they suggested changing the directory
>> from package.mask to package.mask.d, since all you would need to do is
>> rename the file package.mask to something like package.mask/oldmasks and
>> the masks in it would be preserved until you put them in different files
>> in the package.mask directory and removed them from oldmasks, ultimately
>> deleting oldmasks.
>>     
>
> The (one) idea is to keep package.mask as a file for old package managers 
> that don't know about package.* as a directory yet.  Moving package.mask 
> (the file) to package.mask/oldmasks wouldn't accomplish that.   The 
> scenario people are worried about, is where (for example) someone has 
> gone away for their year's service (or two, or whatever) that some 
> nations have, and comes back and tries to upgrade, only to find the tree 
> is so changed that as soon as they sync, their old package manager is 
> broken, to the point they can't use use it to upgrade to a new version, 
> in ordered to be able to upgrade everything else!
>
> Actually splitting the current single file into multiple files isn't 
> really a problem, and would probably be done at the same time package.mask
> (.d) is made a directory, all in the same commit, so it's nothing to 
> worry about. 
>
> That said, in this particular instance, I don't believe that should be a 
> big problem.  Why?  Because portage has supported it for years already, 
> and anyone using a PM that doesn't currently support it has by definition 
> already demonstrated they are reasonably active about their Gentoo based 
> system management choices, and thus isn't likely to let a system go 
> unsynced and the PM unupdated for greater than say 90 days anyway.  Thus, 
> when the policy is approved by council, stick a 90-180 day hold on 
> changing old profiles in the tree, and be done with it.
>
> So ancient PMs shouldn't be a problem either way, here.  Which leaves 
> only a simple bikeshedding issue, whether people prefer keeping the names 
> we have, or switching to the *.d forms in keeping with the pattern used 
> for all those /etc/*.d directories, of which a quick ls here demonstrated 
> there's about double the number I expected, having never actually counted 
> them before.  While I'd vote for sticking with what we have, that /is/ 
> just bikeshedding, and I REALLY want to just get on with it, regardless 
> of what it's named, as the feature really is quite useful.
>
>   

While I like your example, if this were to happen and a couple other
things has been updated, like for example expat a while back and other
similar update nightmares, wouldn't a reinstall be easier and most
likely recommended anyway?  I have seen this recommended and even made
that recommendation on -user a few times.  I would also do a reinstall
on my own system if I had been gone for 6 months or more.   With the
updates coming pretty fast for most things, you would most likely
recompile most everything on the system anyway.  Save the configs and
the world file and just start over from a very fresh stage tarball.

It is good to look out for those braves souls that would try to update
anyway but thought it worth a mention.  Would upgrading be recommended?

< back to my hole >

Dale

:-)  :-) 

Re: [gentoo-dev] Re: package.mask or package.mask.d

[Andrew D Kirch]

Dale wrote:
>
> While I like your example, if this were to happen and a couple other
> things has been updated, like for example expat a while back and other
> similar update nightmares, wouldn't a reinstall be easier and most
> likely recommended anyway?  I have seen this recommended and even made
> that recommendation on -user a few times.  I would also do a reinstall
> on my own system if I had been gone for 6 months or more.   With the
> updates coming pretty fast for most things, you would most likely
> recompile most everything on the system anyway.  Save the configs and
> the world file and just start over from a very fresh stage tarball.
>
> It is good to look out for those braves souls that would try to update
> anyway but thought it worth a mention.  Would upgrading be recommended?
>
> < back to my hole >
>
> Dale
>
> :-)  :-) 
>   

Guys, if your Gentoo install is 6 months old, you're screwed anyway. 
This should really be a non-issue.  I just spent 2 days dealing with
being 3.5 weeks out of date.  Might have been quicker to re-install. 
Ooh well. 

Andrew

[gentoo-dev] Major problems in the tree in the last month

[Torsten Veller]

* Andrew D Kirch <trelane@trelane.net>:
> This should really be a non-issue.  I just spent 2 days dealing with
> being 3.5 weeks out of date.

To help us improve the user experience, what were the problems that
cost you two days?

Re: [gentoo-dev] Major problems in the tree in the last month

[Andrew D Kirch]

Torsten Veller wrote:
> * Andrew D Kirch <trelane@trelane.net>:
>   
>> This should really be a non-issue.  I just spent 2 days dealing with
>> being 3.5 weeks out of date.
>>     
>
> To help us improve the user experience, what were the problems that
> cost you two days?
>
>   


The major problem was getting caught up in the 'kdeprefix' use flag.   
The time loss can basically be explained as:
1. compile kde 4.3 which eventually fails 12+ hours
2. eradicate kde 4.2 and 4.3, 4 or so hours
3. clean up the resulting mess in @world, 8 hours
4. compile kde 4.3 again 12+hours 

This on a dual p4 3ghz takes a bit of time sadly.  I also ran into bug
280443 with XOrg (and have commented appropriately).  It appears dbus
specific files aren't going where they should, and HAL is kicking
unhelpful errors.

Andrew D Kirch
Funtoo.org

Re: [gentoo-dev] Major problems in the tree in the last month

[Jorge Manuel B. S. Vicetto]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew D Kirch wrote:
> Torsten Veller wrote:
>> * Andrew D Kirch <trelane@trelane.net>:
>>   
>>> This should really be a non-issue.  I just spent 2 days dealing with
>>> being 3.5 weeks out of date.
>>>     
>> To help us improve the user experience, what were the problems that
>> cost you two days?
>>
>>   
> 
> 
> The major problem was getting caught up in the 'kdeprefix' use flag.   
> The time loss can basically be explained as:
> 1. compile kde 4.3 which eventually fails 12+ hours
> 2. eradicate kde 4.2 and 4.3, 4 or so hours
> 3. clean up the resulting mess in @world, 8 hours
> 4. compile kde 4.3 again 12+hours 
> 
> This on a dual p4 3ghz takes a bit of time sadly.  I also ran into bug
> 280443 with XOrg (and have commented appropriately).  It appears dbus
> specific files aren't going where they should, and HAL is kicking
> unhelpful errors.
> 
> Andrew D Kirch
> Funtoo.org

As the KDE team Lead and to explain the above, this was an unfortunate
requirement for the KDE team to get KDE-4 in the road to be marked
stable. We should have probably announced it better and warned users
about it sooner.
This only affected users running the testing tree or users that had
already switched to KDE-4 and was required as KDE upstream doesn't
support multiple installs in the same prefix and we had a few nasty
issues with python bindings (not being able to slot them as they install
files into site-dir) and there's a bug in the qt plugin loader that
would make it load plugins from the wrong prefix[1].

 [1] - http://forums.gentoo.org/viewtopic-t-708282.html - FAQ section

- --
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / SPARC / KDE
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkqRLyAACgkQcAWygvVEyALF4ACdEURczaceHoYVjpxwSqqWorhs
EyAAniGbn+JWEgLuMZe1eTO1iUu48WlK
=4nI4
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Major problems in the tree in the last month

[Andrew D Kirch]

Jorge Manuel B. S. Vicetto wrote:
> Andrew D Kirch wrote:
> > Torsten Veller wrote:
> >> * Andrew D Kirch <trelane@trelane.net>:
> >>   
> >>> This should really be a non-issue.  I just spent 2 days dealing with
> >>> being 3.5 weeks out of date.
> >>>     
> >> To help us improve the user experience, what were the problems that
> >> cost you two days?
> >>
> >>   
>
> > The major problem was getting caught up in the 'kdeprefix' use flag.   
> > The time loss can basically be explained as:
> > 1. compile kde 4.3 which eventually fails 12+ hours
> > 2. eradicate kde 4.2 and 4.3, 4 or so hours
> > 3. clean up the resulting mess in @world, 8 hours
> > 4. compile kde 4.3 again 12+hours
>
> > This on a dual p4 3ghz takes a bit of time sadly.  I also ran into bug
> > 280443 with XOrg (and have commented appropriately).  It appears dbus
> > specific files aren't going where they should, and HAL is kicking
> > unhelpful errors.
>
> > Andrew D Kirch
> > Funtoo.org
>
> As the KDE team Lead and to explain the above, this was an unfortunate
> requirement for the KDE team to get KDE-4 in the road to be marked
> stable. We should have probably announced it better and warned users
> about it sooner.
> This only affected users running the testing tree or users that had
> already switched to KDE-4 and was required as KDE upstream doesn't
> support multiple installs in the same prefix and we had a few nasty
> issues with python bindings (not being able to slot them as they install
> files into site-dir) and there's a bug in the qt plugin loader that
> would make it load plugins from the wrong prefix[1].
>
>  [1] - http://forums.gentoo.org/viewtopic-t-708282.html - FAQ section
>
It wasn't a big deal.  Sadly KDE takes darn near forever to compile. 
(or at least seems like it).  If you get into a situation where you
compile it twice that's at least a day lost.

Andrew

[gentoo-dev] Re: package.mask or package.mask.d

[Duncan]

Andrew D Kirch posted on Sun, 23 Aug 2009 04:05:03 -0400 as excerpted:

> Dale wrote:
>>
>> While I like your example, if this were to happen and a couple other
>> things has been updated, like for example expat a while back and other
>> similar update nightmares, wouldn't a reinstall be easier and most
>> likely recommended anyway?  I have seen this recommended and even made
>> that recommendation on -user a few times.  I would also do a reinstall
>> on my own system if I had been gone for 6 months or more.   With the
>> updates coming pretty fast for most things, you would most likely
>> recompile most everything on the system anyway.  Save the configs and
>> the world file and just start over from a very fresh stage tarball.

I thought that's what I argued, a bit later -- that at a year or more out 
of date, there's enough going to be recompiled anyway, might as well just 
do the reinstall from the latest stages, which are at least reasonably 
well used and thus reasonably well debugged, rather than trying to 
upgrade from a year or more out, something that's much rarer and thus 
likely to be a major headache, even if there's nothing deliberately done 
to kill it and no known direct incompatibilities.

And I've used the six-month figure myself.  This is what I usually say:

I try to do it once a week at least (twice a week is better), to keep the 
changes to something manageable, even when there's a several-hundred-
package kde upgrade. =:^P  But I can see the case being made for once a 
month, which would trade a few less updates (intermediate updates you 
didn't see) for a bigger headache trying to manage it all, particularly 
if something goes wrong.  And for those on a routine monthly update 
schedule, it's plausible something could delay updating a month or two, 
thus making it three months.  Heh, that's some people's vacation.

But at three months, I'd be beginning to consider a reinstall from 
stages, tho I'd probably still do the in-place upgrade.  But beyond that, 
people are really only making it harder on themselves, and by six months, 
I really do believe the stages method will be both cleaner and easier, 
tho I could still see people trying the in-place method but I doubt I 
would.  But at a year, honestly, what /are/ people thinking, trying to do 
the in-place upgrade?  Well, I suppose it wasn't that long ago Gentoo 
releng was having problems, and the install media /was/ over a year old, 
but even then, at least upgrading from it would be done decently 
frequently and thus be decently tested, unlike trying to upgrade in place 
from some arbitrary version.  But Gentoo's doing rather better in that 
regard now, and with automated weekly stages, why on /earth/ would 
someone put themselves thru the trouble of an in-place upgrade at a year 
out, when they can do it with the far better tested weekly stages, and 
just by doing that, they'll be at worst a few weeks out (if there was 
some blocker and the stages for their arch hadn't built properly for a 
couple weeks).

> Guys, if your Gentoo install is 6 months old, you're screwed anyway.
> This should really be a non-issue.  I just spent 2 days dealing with
> being 3.5 weeks out of date.  Might have been quicker to re-install. Ooh
> well.

Probably not, for 3.5 weeks.  Even if that 3.5 weeks is over a big 
upgrade, like a new xorg, or kde/gnome/xfce, whatever you run.  At that 
point, it'll be getting close, but there's still going to be major parts 
of the system that are still upto date.  But beyond 3 months, it's 
questionable, and beyond six, yeah, just go the stages route again, it'll 
be less hassle.

Actually, new thought here!

How close you are to the USE/C/CXX/LD- flags the stages are built with 
could make a big difference, now.  If you're running identical flags (or 
close enough to it you can consider this), then for the core stages 
packages, you can simply install the stages every week, and get at least 
somewhat tested "near free" upgrades just doing that.  If you know the 
update cycle, and grab the stages the day /before/ they update, you've 
just gotten yourself six days' worth of "free" testing of the exact same 
packages you're using, in /addition/ to the fact that you just saved 
yourself a bit of compiling.

I hadn't thought of /that/ angle before.  Interesting idea! =:^)  It's 
been long enough since I installed (I've been doing rolling updates since 
my initial install of 2004.1) that I'm not sure how /practical/ the idea 
might be (are most of the packages in the tarball "limited build", thus 
needing rebuilt anyway?), but if they're being built weekly now... it 
really /is/ an interesting idea!

But not for me, of course, as like many Gentooers, I have my own ideas 
about all those flags, and using distribution defaults brings back bad 
memories of the binary based distributions...  But maybe for /somebody/.  
The /theory/'s there.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: package.mask or package.mask.d

[Dale]

Duncan wrote:
> Andrew D Kirch posted on Sun, 23 Aug 2009 04:05:03 -0400 as excerpted:
>
>   
>> Dale wrote:
>>     
>>> While I like your example, if this were to happen and a couple other
>>> things has been updated, like for example expat a while back and other
>>> similar update nightmares, wouldn't a reinstall be easier and most
>>> likely recommended anyway?  I have seen this recommended and even made
>>> that recommendation on -user a few times.  I would also do a reinstall
>>> on my own system if I had been gone for 6 months or more.   With the
>>> updates coming pretty fast for most things, you would most likely
>>> recompile most everything on the system anyway.  Save the configs and
>>> the world file and just start over from a very fresh stage tarball.
>>>       
>
> I thought that's what I argued, a bit later -- that at a year or more out 
> of date, there's enough going to be recompiled anyway, might as well just 
> do the reinstall from the latest stages, which are at least reasonably 
> well used and thus reasonably well debugged, rather than trying to 
> upgrade from a year or more out, something that's much rarer and thus 
> likely to be a major headache, even if there's nothing deliberately done 
> to kill it and no known direct incompatibilities.
>
> And I've used the six-month figure myself.  This is what I usually say:
>
> I try to do it once a week at least (twice a week is better), to keep the 
> changes to something manageable, even when there's a several-hundred-
> package kde upgrade. =:^P  But I can see the case being made for once a 
> month, which would trade a few less updates (intermediate updates you 
> didn't see) for a bigger headache trying to manage it all, particularly 
> if something goes wrong.  And for those on a routine monthly update 
> schedule, it's plausible something could delay updating a month or two, 
> thus making it three months.  Heh, that's some people's vacation.
>
> But at three months, I'd be beginning to consider a reinstall from 
> stages, tho I'd probably still do the in-place upgrade.  But beyond that, 
> people are really only making it harder on themselves, and by six months, 
> I really do believe the stages method will be both cleaner and easier, 
> tho I could still see people trying the in-place method but I doubt I 
> would.  But at a year, honestly, what /are/ people thinking, trying to do 
> the in-place upgrade?  Well, I suppose it wasn't that long ago Gentoo 
> releng was having problems, and the install media /was/ over a year old, 
> but even then, at least upgrading from it would be done decently 
> frequently and thus be decently tested, unlike trying to upgrade in place 
> from some arbitrary version.  But Gentoo's doing rather better in that 
> regard now, and with automated weekly stages, why on /earth/ would 
> someone put themselves thru the trouble of an in-place upgrade at a year 
> out, when they can do it with the far better tested weekly stages, and 
> just by doing that, they'll be at worst a few weeks out (if there was 
> some blocker and the stages for their arch hadn't built properly for a 
> couple weeks).
>
>   
>> Guys, if your Gentoo install is 6 months old, you're screwed anyway.
>> This should really be a non-issue.  I just spent 2 days dealing with
>> being 3.5 weeks out of date.  Might have been quicker to re-install. Ooh
>> well.
>>     
>
> Probably not, for 3.5 weeks.  Even if that 3.5 weeks is over a big 
> upgrade, like a new xorg, or kde/gnome/xfce, whatever you run.  At that 
> point, it'll be getting close, but there's still going to be major parts 
> of the system that are still upto date.  But beyond 3 months, it's 
> questionable, and beyond six, yeah, just go the stages route again, it'll 
> be less hassle.
>
> Actually, new thought here!
>
> How close you are to the USE/C/CXX/LD- flags the stages are built with 
> could make a big difference, now.  If you're running identical flags (or 
> close enough to it you can consider this), then for the core stages 
> packages, you can simply install the stages every week, and get at least 
> somewhat tested "near free" upgrades just doing that.  If you know the 
> update cycle, and grab the stages the day /before/ they update, you've 
> just gotten yourself six days' worth of "free" testing of the exact same 
> packages you're using, in /addition/ to the fact that you just saved 
> yourself a bit of compiling.
>
> I hadn't thought of /that/ angle before.  Interesting idea! =:^)  It's 
> been long enough since I installed (I've been doing rolling updates since 
> my initial install of 2004.1) that I'm not sure how /practical/ the idea 
> might be (are most of the packages in the tarball "limited build", thus 
> needing rebuilt anyway?), but if they're being built weekly now... it 
> really /is/ an interesting idea!
>
> But not for me, of course, as like many Gentooers, I have my own ideas 
> about all those flags, and using distribution defaults brings back bad 
> memories of the binary based distributions...  But maybe for /somebody/.  
> The /theory/'s there.
>
>   

I think we agree more than disagree.  It mostly depends on what updates
there has been and if the fancy new portage can get past any blockages
and make the needed fixes.  I guess whether it is a year, six months or
even three months depends on what has been updated, what has to be
recompiled and how difficult it is to configure them afterwards.  Right
now, after the recent python upgrade, I'm recompiling over 60 packages
including OOo.  If it meant recompiling most of KDE, then a reinstall
may be faster.

I'm just not sure that portage should be soooo backward compatible as it
appears to be now.

Dale

:-)  :-) 

Re: [gentoo-dev] Re: package.mask or package.mask.d

[Andrew D Kirch]

Dale wrote:
>
> I'm just not sure that portage should be soooo backward compatible as it
> appears to be now.
>   
Dale,

It really doesn't have to, unless you're not running portage... in which
case we have to wait on all package managers to implement every major
feature change the others want.   The result of this was PMS and EAPIs.
  These are an effort to define feature sets the PM's can agree on.

Interestingly some package managers, or perhaps their developers, are
louder and more abrasive than others.  This is causing an overall
reduction in the implementation speed of these EAPI's and quite a bit of
confusion with 10.0, along with much wharrgarbl and FUD.  It also
consumed nearly the entire agenda of the previous Gentoo Council. 
Things are getting better though.

Andrew D Kirch
Funtoo.org

Re: [gentoo-dev] Re: package.mask or package.mask.d

[Gilles Dartiguelongue]

Le dimanche 23 août 2009 à 10:24 +0000, Duncan a écrit :
> But at three months, I'd be beginning to consider a reinstall from 
> stages, tho I'd probably still do the in-place upgrade.  But beyond
> that, 
> people are really only making it harder on themselves, and by six
> months, 

It really depends on the packages you are using. I've updated machines
over 6 month old period (which got to this state for whatever reason,
doesn't matter) and it was just a breeze to update because there wasn't
that much packages that changed in an "upgrade blocker" way. Plus all
the hard work was done on the binary building machine. It would have
been much longer to re-install 10+ machines. So please don't generalize
your experience with fast moving packages to the tree.

-- 
Gilles Dartiguelongue <eva@gentoo.org>
Gentoo