[gentoo-dev] POSIX capability in Gentoo

[gentoo-dev] POSIX capability in Gentoo

[Anthony G. Basile]

Hi everyone,

A couple of days ago, bonsaikitten (Patrick), kerframil (Kerin Millar)
and myself were talking about other distros moving away from setuid
binaries towards caps.  Openwall and Fedora are now setuid-less [1]. 
Some googling showed that Constanze has done quite a bit of work in the
area and that there was a consensus to include functions to set caps
within portage [2].  I don't know what, if anything has been done since
then, but I'd like to lend my support.

Ref
[1] http://lwn.net/Articles/420969/
[2] http://www.gossamer-threads.com/lists/gentoo/dev/226948

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 8040 5A4D 8709 21B1 1A88  33CE 979C AF40 D045 5535
GnuPG ID  : D0455535

Re: [gentoo-dev] POSIX capability in Gentoo

[Nirbheek Chauhan]

On Sun, Jul 31, 2011 at 8:13 PM, Anthony G. Basile <blueness@gentoo.org> wrote:
> Hi everyone,
>
> A couple of days ago, bonsaikitten (Patrick), kerframil (Kerin Millar)
> and myself were talking about other distros moving away from setuid
> binaries towards caps.  Openwall and Fedora are now setuid-less [1].
> Some googling showed that Constanze has done quite a bit of work in the
> area and that there was a consensus to include functions to set caps
> within portage [2].  I don't know what, if anything has been done since
> then, but I'd like to lend my support.
>

One problem that came up was that a lot of people use tmpfs for
/var/tmp/portage, and tmpfs doesn't support xattrs which are needed
for setting caps.

Linux 3.0 has added support for xattrs with tmpfs (the redhat folks
did the work, afaik), so that problem is partly solved now.


-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] POSIX capability in Gentoo

[Anthony G. Basile]

On 07/31/2011 03:46 PM, Nirbheek Chauhan wrote:
> On Sun, Jul 31, 2011 at 8:13 PM, Anthony G. Basile <blueness@gentoo.org> wrote:
>> Hi everyone,
>>
>> A couple of days ago, bonsaikitten (Patrick), kerframil (Kerin Millar)
>> and myself were talking about other distros moving away from setuid
>> binaries towards caps.  Openwall and Fedora are now setuid-less [1].
>> Some googling showed that Constanze has done quite a bit of work in the
>> area and that there was a consensus to include functions to set caps
>> within portage [2].  I don't know what, if anything has been done since
>> then, but I'd like to lend my support.
>>
> One problem that came up was that a lot of people use tmpfs for
> /var/tmp/portage, and tmpfs doesn't support xattrs which are needed
> for setting caps.
>
> Linux 3.0 has added support for xattrs with tmpfs (the redhat folks
> did the work, afaik), so that problem is partly solved now.
>
>

I know, there are lots of places where xattrs is not supported that lead
to the same problem.  I'm tempted to respond with pkg_postinst() but I
see QA problems written all over that.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 8040 5A4D 8709 21B1 1A88  33CE 979C AF40 D045 5535
GnuPG ID  : D0455535

Re: [gentoo-dev] POSIX capability in Gentoo

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 294 bytes --]

On Sun, 31 Jul 2011 22:28:35 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> Will packages always explicitly set caps themselves or will sometimes
> upstream do that for us?

I've no doubt some upstreams will try... But userpriv should stop most
of the damage.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] POSIX capability in Gentoo

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

On Mon, 1 Aug 2011 01:16:21 +0530
Nirbheek Chauhan <nirbheek@gentoo.org> wrote:

> On Sun, Jul 31, 2011 at 8:13 PM, Anthony G. Basile
> <blueness@gentoo.org> wrote:
> > Hi everyone,
> >
> > A couple of days ago, bonsaikitten (Patrick), kerframil (Kerin
> > Millar) and myself were talking about other distros moving away
> > from setuid binaries towards caps.  Openwall and Fedora are now
> > setuid-less [1]. Some googling showed that Constanze has done quite
> > a bit of work in the area and that there was a consensus to include
> > functions to set caps within portage [2].  I don't know what, if
> > anything has been done since then, but I'd like to lend my support.
> >
> 
> One problem that came up was that a lot of people use tmpfs for
> /var/tmp/portage, and tmpfs doesn't support xattrs which are needed
> for setting caps.

Will packages always explicitly set caps themselves or will sometimes
upstream do that for us?

IOW, will we have total control over actual caps?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] POSIX capability in Gentoo

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1521 bytes --]

On Sun, 31 Jul 2011 16:00:40 -0400
"Anthony G. Basile" <blueness@gentoo.org> wrote:

> On 07/31/2011 03:46 PM, Nirbheek Chauhan wrote:
> > On Sun, Jul 31, 2011 at 8:13 PM, Anthony G. Basile
> > <blueness@gentoo.org> wrote:
> >> Hi everyone,
> >>
> >> A couple of days ago, bonsaikitten (Patrick), kerframil (Kerin
> >> Millar) and myself were talking about other distros moving away
> >> from setuid binaries towards caps.  Openwall and Fedora are now
> >> setuid-less [1]. Some googling showed that Constanze has done
> >> quite a bit of work in the area and that there was a consensus to
> >> include functions to set caps within portage [2].  I don't know
> >> what, if anything has been done since then, but I'd like to lend
> >> my support.
> >>
> > One problem that came up was that a lot of people use tmpfs for
> > /var/tmp/portage, and tmpfs doesn't support xattrs which are needed
> > for setting caps.
> >
> > Linux 3.0 has added support for xattrs with tmpfs (the redhat folks
> > did the work, afaik), so that problem is partly solved now.
> 
> I know, there are lots of places where xattrs is not supported that
> lead to the same problem.  I'm tempted to respond with pkg_postinst()
> but I see QA problems written all over that.

We can either do that or 'Future EAPI' capsetting in PMS. Then, a PM
could implement capsetting functions in a such way that they will
preserve caps internally to PM and re-set them when merging to livefs.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] POSIX capability in Gentoo

[Anthony G. Basile]

On 08/02/2011 03:08 AM, Michał Górny wrote:
> On Sun, 31 Jul 2011 16:00:40 -0400
> "Anthony G. Basile" <blueness@gentoo.org> wrote:
> 
>> On 07/31/2011 03:46 PM, Nirbheek Chauhan wrote:
>>> On Sun, Jul 31, 2011 at 8:13 PM, Anthony G. Basile
>>> <blueness@gentoo.org> wrote:
>>>> Hi everyone,
>>>>
>>>> A couple of days ago, bonsaikitten (Patrick), kerframil (Kerin
>>>> Millar) and myself were talking about other distros moving away
>>>> from setuid binaries towards caps.  Openwall and Fedora are now
>>>> setuid-less [1]. Some googling showed that Constanze has done
>>>> quite a bit of work in the area and that there was a consensus to
>>>> include functions to set caps within portage [2].  I don't know
>>>> what, if anything has been done since then, but I'd like to lend
>>>> my support.
>>>>
>>> One problem that came up was that a lot of people use tmpfs for
>>> /var/tmp/portage, and tmpfs doesn't support xattrs which are needed
>>> for setting caps.
>>>
>>> Linux 3.0 has added support for xattrs with tmpfs (the redhat folks
>>> did the work, afaik), so that problem is partly solved now.
>>
>> I know, there are lots of places where xattrs is not supported that
>> lead to the same problem.  I'm tempted to respond with pkg_postinst()
>> but I see QA problems written all over that.
> 
> We can either do that or 'Future EAPI' capsetting in PMS. Then, a PM
> could implement capsetting functions in a such way that they will
> preserve caps internally to PM and re-set them when merging to livefs.
> 

I prefer capsetting in the PMS itself, with a nice clean function which
auto detects all the necessary conditions and transparently preserves
caps, as you suggest.  Maybe this can be in EAPI=5.

I'm also wondering if, in the mean time, it might be worth writing a
bash script and/or howto on converting as many binaries as possible from
setuid to caps --- hitting up all the usual suspects.  Its not ideal but
might still be useful until we get this squarely in the PMS.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 8040 5A4D 8709 21B1 1A88  33CE 979C AF40 D045 5535
GnuPG ID  : D0455535

Re: [gentoo-dev] POSIX capability in Gentoo

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

On Tue, 02 Aug 2011 10:28:58 -0400
"Anthony G. Basile" <blueness@gentoo.org> wrote:
> I prefer capsetting in the PMS itself, with a nice clean function
> which auto detects all the necessary conditions and transparently
> preserves caps, as you suggest.  Maybe this can be in EAPI=5.

Would need a spec, along with a way of dealing with all the problems:
what happens if the build fs supports caps but the install fs doesn't?
What about if caps are supported on both but in different ways (tmpfs
on some kernels)? Is it up to the PM to deal with that? How does the PM
even know?

> I'm also wondering if, in the mean time, it might be worth writing a
> bash script and/or howto on converting as many binaries as possible
> from setuid to caps --- hitting up all the usual suspects.  Its not
> ideal but might still be useful until we get this squarely in the PMS.

PMS currently explicitly states that caps might get clobbered on a
merge (because Portage does that sometimes). So if you're doing it now,
it'd have to be as a pkg_postinst thing. But I'd strongly recommend not
going that route, since it'll almost certainly go horribly wrong in a
"your system randomly no longer works" kind of way... Better to ban
things from using caps for now.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] POSIX capability in Gentoo

[Anthony G. Basile]

On 08/02/2011 10:31 AM, Ciaran McCreesh wrote:
> On Tue, 02 Aug 2011 10:28:58 -0400
> "Anthony G. Basile" <blueness@gentoo.org> wrote:
>> I prefer capsetting in the PMS itself, with a nice clean function
>> which auto detects all the necessary conditions and transparently
>> preserves caps, as you suggest.  Maybe this can be in EAPI=5.
> Would need a spec, along with a way of dealing with all the problems:
> what happens if the build fs supports caps but the install fs doesn't?
> What about if caps are supported on both but in different ways (tmpfs
> on some kernels)? Is it up to the PM to deal with that? How does the PM
> even know?
>

That's exactly what I was thinking of for the PM.  It would have to
autodetect all that.  Eg. it could create a test file on each fs and
then do a getcap on it and if it fails, you have your answer.  If
necessary and it exists, it could look at /proc/config.  I think it's
doable.

>> I'm also wondering if, in the mean time, it might be worth writing a
>> bash script and/or howto on converting as many binaries as possible
>> from setuid to caps --- hitting up all the usual suspects.  Its not
>> ideal but might still be useful until we get this squarely in the PMS.
> PMS currently explicitly states that caps might get clobbered on a
> merge (because Portage does that sometimes). So if you're doing it now,
> it'd have to be as a pkg_postinst thing. But I'd strongly recommend not
> going that route, since it'll almost certainly go horribly wrong in a
> "your system randomly no longer works" kind of way... Better to ban
> things from using caps for now.
>

I was thinking something even dirtier, something outside of the PMS
altogether, along the lines of what one does when converting to a
selinux system where one relabels the entire filesystem with rlpkg.  So
no, not something via pkg_postinst().

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 8040 5A4D 8709 21B1 1A88  33CE 979C AF40 D045 5535
GnuPG ID  : D0455535

Re: [gentoo-dev] POSIX capability in Gentoo

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2021 bytes --]

On Tue, 02 Aug 2011 10:51:22 -0400
"Anthony G. Basile" <blueness@gentoo.org> wrote:
> > Would need a spec, along with a way of dealing with all the
> > problems: what happens if the build fs supports caps but the
> > install fs doesn't? What about if caps are supported on both but in
> > different ways (tmpfs on some kernels)? Is it up to the PM to deal
> > with that? How does the PM even know?
> 
> That's exactly what I was thinking of for the PM.  It would have to
> autodetect all that.

That's the problematic part... It's not quite "the PM just needs to
come up with a cure for cancer", but it's decidedly non-trivial.

> Eg. it could create a test file on each fs and
> then do a getcap on it and if it fails, you have your answer.

But it can and will be merging to multiple filesystems, some of which
support caps and some of which don't.

Maybe the answer is to have the PM do the merge, including caps, and if
it detects that the caps setting failed then it should fall back to
some kind of set*id bit (but which one?). But I'm not sure that setting
caps that won't actually work will necessarily give a failure.

Another possibility is to simply require that the PM preserve caps from
the build fs to the root fs, and if it fails, to abort horribly (except
we hate dying mid-merge, since it's impossible to clean up). Then it's
the user's responsibility to turn off caps on their build fs if
necessary.

But neither of those are anywhere close to implementable without a lot
of careful thought and planning... We need to *prove* that we're safe
here, not guess that we're probably ok based upon a bit of testing.

And we haven't even started talking about binaries yet...

> I was thinking something even dirtier, something outside of the PMS
> altogether, along the lines of what one does when converting to a
> selinux system where one relabels the entire filesystem with rlpkg.
> So no, not something via pkg_postinst().

Please don't.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] POSIX capability in Gentoo

[Anthony G. Basile]

On 08/02/2011 10:54 AM, Ciaran McCreesh wrote:
>> > I was thinking something even dirtier, something outside of the PMS
>> > altogether, along the lines of what one does when converting to a
>> > selinux system where one relabels the entire filesystem with rlpkg.
>> > So no, not something via pkg_postinst().
> Please don't.
>

Why would this be bad?

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 8040 5A4D 8709 21B1 1A88  33CE 979C AF40 D045 5535
GnuPG ID  : D0455535

Re: [gentoo-dev] POSIX capability in Gentoo

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 808 bytes --]

On Tue, 02 Aug 2011 11:05:34 -0400
"Anthony G. Basile" <blueness@gentoo.org> wrote:
> On 08/02/2011 10:54 AM, Ciaran McCreesh wrote:
> >> > I was thinking something even dirtier, something outside of the
> >> > PMS altogether, along the lines of what one does when converting
> >> > to a selinux system where one relabels the entire filesystem
> >> > with rlpkg. So no, not something via pkg_postinst().
>
> > Please don't.
> 
> Why would this be bad?

Because going behind the package mangler's back results in horribly
screwed up systems (as anyone who's ever used lafilefixer will tell
you...).

This is something worth solving properly, and because of all the damage
it can cause if done badly, it's worth not doing at all until a proper
solution is available.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] POSIX capability in Gentoo

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1543 bytes --]

On Tue, 02 Aug 2011 10:51:22 -0400
"Anthony G. Basile" <blueness@gentoo.org> wrote:

> On 08/02/2011 10:31 AM, Ciaran McCreesh wrote:
> > On Tue, 02 Aug 2011 10:28:58 -0400
> > "Anthony G. Basile" <blueness@gentoo.org> wrote:
> >> I prefer capsetting in the PMS itself, with a nice clean function
> >> which auto detects all the necessary conditions and transparently
> >> preserves caps, as you suggest.  Maybe this can be in EAPI=5. 
> > Would need a spec, along with a way of dealing with all the
> > problems: what happens if the build fs supports caps but the
> > install fs doesn't? What about if caps are supported on both but in
> > different ways (tmpfs on some kernels)? Is it up to the PM to deal
> > with that? How does the PM even know?
> >
> 
> That's exactly what I was thinking of for the PM.  It would have to
> autodetect all that.  Eg. it could create a test file on each fs and
> then do a getcap on it and if it fails, you have your answer.  If
> necessary and it exists, it could look at /proc/config.  I think it's
> doable.

Just let the capsetting function store all details internally when
called. I don't think it's really important whether build fs capsetting
succeeds. So, it's like:

1) capset on buildfs, store details internally;
2) move to livefs;
3) [optionally] getcap on livefs, done if set;
4) capset on livefs;
5) getcap on livefs, done if set;
6) fallback to set?id (using info from stored capsetting function call)
   if necessary.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] POSIX capability in Gentoo

[Rich Freeman]

On Tue, Aug 2, 2011 at 11:05 AM, Anthony G. Basile <blueness@gentoo.org> wrote:
> On 08/02/2011 10:54 AM, Ciaran McCreesh wrote:
>>> > I was thinking something even dirtier, something outside of the PMS
>>> > altogether, along the lines of what one does when converting to a
>>> > selinux system where one relabels the entire filesystem with rlpkg.
>>> > So no, not something via pkg_postinst().
>> Please don't.
> Why would this be bad?

Something that comes to mind would be the inability to systematically
verify the installed system.  We obviously don't currently store posix
capabilities the way we store mtimes and hashes, but I would think
that this would just be one more part of the EAPI if we properly
define it.

That said, I don't see manual scripts outside of portage being a
possible workaround, but it should probably only be used
experimentally.

Rich

Re: [gentoo-dev] POSIX capability in Gentoo

[Anthony G. Basile]

On 08/02/2011 11:05 AM, Ciaran McCreesh wrote:
>>> Please don't.
>> > 
>> > Why would this be bad?
> Because going behind the package mangler's back results in horribly
> screwed up systems (as anyone who's ever used lafilefixer will tell
> you...).

Is rlpkg going behind the PM's back when it does selinux labelings?  I
know there are difference, but if there's a screwup in some policy, it
also leads to horribly screwed up system.  Nonetheless, I'm not
insensitive to what you are saying, and I think the safer approach would
be to write a howto and show the user how to manually convert some
typical binaries.  There are only a handful that would be targeted.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 8040 5A4D 8709 21B1 1A88  33CE 979C AF40 D045 5535
GnuPG ID  : D0455535

Re: [gentoo-dev] POSIX capability in Gentoo

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 765 bytes --]

On Tue, 02 Aug 2011 11:19:21 -0400
"Anthony G. Basile" <blueness@gentoo.org> wrote:
> Is rlpkg going behind the PM's back when it does selinux labelings?

Yup. Also, note that PMS has wording for selinux.

> I know there are difference, but if there's a screwup in some policy, it
> also leads to horribly screwed up system.  Nonetheless, I'm not
> insensitive to what you are saying, and I think the safer approach
> would be to write a howto and show the user how to manually convert
> some typical binaries.  There are only a handful that would be
> targeted.

Why are caps so important that we should be encouraging users to
subvert things, yet at the same time not important enough that they
should be handled properly?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: POSIX capability in Gentoo

[Duncan]

Ciaran McCreesh posted on Tue, 02 Aug 2011 16:05:54 +0100 as excerpted:

> Because going behind the package mangler's back results in horribly
> screwed up systems (as anyone who's ever used lafilefixer will tell
> you...).

Well, not "anyone".  I never had any problems with it.

(YMMV, but soon enough, I switched to an installmask with an exception 
for libtool, then rebuilt the system.  No *.la file worries since! =:^)

(Observation: Unqualified any/all statements are rather like greedy .* 
regex handling, sometimes they include more than one might intend!)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: POSIX capability in Gentoo

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 908 bytes --]

On Tue, 2 Aug 2011 17:11:28 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:
> Ciaran McCreesh posted on Tue, 02 Aug 2011 16:05:54 +0100 as
> excerpted:
> > Because going behind the package mangler's back results in horribly
> > screwed up systems (as anyone who's ever used lafilefixer will tell
> > you...).
> 
> Well, not "anyone".  I never had any problems with it.

You did, you just didn't notice it. You'll find out sooner or later
when you get bitten by one of the will-never-be-uninstalled-now .la
files that it modified on your system without updating VDB.

> (Observation: Unqualified any/all statements are rather like
> greedy .* regex handling, sometimes they include more than one might
> intend!)

Well, if you prefer, "anyone who's ever used lafilefixer and then either
looked carefully at what happened or got hit by random nastiness later
on".

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: POSIX capability in Gentoo

[Jonathan Callen]

Ciaran McCreesh wrote:

> On Tue, 2 Aug 2011 17:11:28 +0000 (UTC)
> Duncan <1i5t5.duncan@cox.net> wrote:
>> Ciaran McCreesh posted on Tue, 02 Aug 2011 16:05:54 +0100 as
>> excerpted:
>> > Because going behind the package mangler's back results in horribly
>> > screwed up systems (as anyone who's ever used lafilefixer will tell
>> > you...).
>> 
>> Well, not "anyone".  I never had any problems with it.
> 
> You did, you just didn't notice it. You'll find out sooner or later
> when you get bitten by one of the will-never-be-uninstalled-now .la
> files that it modified on your system without updating VDB.
> 
>> (Observation: Unqualified any/all statements are rather like
>> greedy .* regex handling, sometimes they include more than one might
>> intend!)
> 
> Well, if you prefer, "anyone who's ever used lafilefixer and then either
> looked carefully at what happened or got hit by random nastiness later
> on".
> 

That statement needs one more qualification: "and doesn't use portage".  
Portage will (by default) remove files on uninstall even if they *do not* 
match the checksum recorded in the vdb.  This implies that most people will 
*not* see any issues due to something other than the package manager 
modifying the files behind the package manager's back.
-- 
Jonathan Callen

Re: [gentoo-dev] Re: POSIX capability in Gentoo

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 635 bytes --]

On Tue, 02 Aug 2011 13:36:12 -0400
Jonathan Callen <abcd@gentoo.org> wrote:
> That statement needs one more qualification: "and doesn't use
> portage". Portage will (by default) remove files on uninstall even if
> they *do not* match the checksum recorded in the vdb.  This implies
> that most people will *not* see any issues due to something other
> than the package manager modifying the files behind the package
> manager's back.

Ugh, seriously? When did that happen? That's a massive change to how
VDB is supposed to work.

Maybe we need to spec VDB after all to avoid that kind of nonsense.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: POSIX capability in Gentoo

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 837 bytes --]

2011-08-02 19:39:18 Ciaran McCreesh napisał(a):
> On Tue, 02 Aug 2011 13:36:12 -0400
> Jonathan Callen <abcd@gentoo.org> wrote:
> > That statement needs one more qualification: "and doesn't use
> > portage". Portage will (by default) remove files on uninstall even if
> > they *do not* match the checksum recorded in the vdb.  This implies
> > that most people will *not* see any issues due to something other
> > than the package manager modifying the files behind the package
> > manager's back.
> 
> Ugh, seriously? When did that happen?

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=a133cb89d5279df7febcd0c8ab3890e2ccfb897a
 
> Maybe we need to spec VDB after all to avoid that kind of nonsense.

I think that unmerge-orphans is a useful feature.

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: POSIX capability in Gentoo

[Brian Harring]

On Tue, Aug 02, 2011 at 06:39:18PM +0100, Ciaran McCreesh wrote:
> On Tue, 02 Aug 2011 13:36:12 -0400
> Jonathan Callen <abcd@gentoo.org> wrote:
> > That statement needs one more qualification: "and doesn't use
> > portage". Portage will (by default) remove files on uninstall even if
> > they *do not* match the checksum recorded in the vdb.  This implies
> > that most people will *not* see any issues due to something other
> > than the package manager modifying the files behind the package
> > manager's back.
> 
> Ugh, seriously? When did that happen? That's a massive change to how
> VDB is supposed to work.

That's been in place a long while; pkgcore has done it from day one 
also.

That's not a "massive change" to vdb behaviour either; file collisions 
aren't supposed to occur, as such ownership of the file is basically 
guranteed back to a single package.  Throw in CONFIG_PROTECT for 
adjusting the behaviour, and you have a far more preferable norm than 
"lets just leave a shit ton of .pyc/.pyo on the fs".

Moving on...
~brian

[gentoo-dev] Re: POSIX capability in Gentoo

[Duncan]

Arfrever Frehtes Taifersar Arahesis posted on Tue, 02 Aug 2011 22:46:54
+0200 as excerpted:

> 2011-08-02 19:39:18 Ciaran McCreesh napisał(a):
>> On Tue, 02 Aug 2011 13:36:12 -0400 Jonathan Callen <abcd@gentoo.org>
>> wrote:
>> > That statement needs one more qualification: "and doesn't use
>> > portage". Portage will (by default) remove files on uninstall even if
>> > they *do not* match the checksum recorded in the vdb.  This implies
>> > that most people will *not* see any issues due to something other
>> > than the package manager modifying the files behind the package
>> > manager's back.
>> 
>> Ugh, seriously? When did that happen?
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/
portage.git;a=commit;h=a133cb89d5279df7febcd0c8ab3890e2ccfb897a
>  
>> Maybe we need to spec VDB after all to avoid that kind of nonsense.
> 
> I think that unmerge-orphans is a useful feature.

Indeed.  FEATURES=unmerge-orphans is optional which is good, but I'm glad 
it's there.  I've no idea what the default is as I've had that on ever 
since I saw the changelog entry where it was introduced.

That'd likely explain why I never had problems with lafilefixer tho.  I'd 
guess the unmerge-orphans feature and lafilefixer appeared about the same 
time, at least for ~arch.

Of course, I have FEATURES=fixlafiles set too, so it'd be handled by 
portage automatically now if I didn't have (PKG_)INSTALL_MASK="*.la" 
killing them but for libtool itself.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: POSIX capability in Gentoo

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1368 bytes --]

On Tue, 2 Aug 2011 17:29:29 -0700
Brian Harring <ferringb@gmail.com> wrote:
> On Tue, Aug 02, 2011 at 06:39:18PM +0100, Ciaran McCreesh wrote:
> > On Tue, 02 Aug 2011 13:36:12 -0400
> > Jonathan Callen <abcd@gentoo.org> wrote:
> > > That statement needs one more qualification: "and doesn't use
> > > portage". Portage will (by default) remove files on uninstall
> > > even if they *do not* match the checksum recorded in the vdb.
> > > This implies that most people will *not* see any issues due to
> > > something other than the package manager modifying the files
> > > behind the package manager's back.
> > 
> > Ugh, seriously? When did that happen? That's a massive change to how
> > VDB is supposed to work.
> 
> That's been in place a long while; pkgcore has done it from day one 
> also.
> 
> That's not a "massive change" to vdb behaviour either; file
> collisions aren't supposed to occur, as such ownership of the file is
> basically guranteed back to a single package.  Throw in
> CONFIG_PROTECT for adjusting the behaviour, and you have a far more
> preferable norm than "lets just leave a shit ton of .pyc/.pyo on the
> fs".

It is a massive change, since if the feature is there then people don't
feel bad about writing lousy pkg_ functions that leave a load
of .pyc / .pyo files all over the place.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: POSIX capability in Gentoo

[Brian Harring]

On Wed, Aug 03, 2011 at 12:34:21PM +0100, Ciaran McCreesh wrote:
> On Tue, 2 Aug 2011 17:29:29 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> > That's not a "massive change" to vdb behaviour either; file
> > collisions aren't supposed to occur, as such ownership of the file is
> > basically guranteed back to a single package.  Throw in
> > CONFIG_PROTECT for adjusting the behaviour, and you have a far more
> > preferable norm than "lets just leave a shit ton of .pyc/.pyo on the
> > fs".
> 
> It is a massive change, since if the feature is there then people don't
> feel bad about writing lousy pkg_ functions that leave a load
> of .pyc / .pyo files all over the place.

Quoting the good spec:

"The unmerge process removes an installed package's files. It is not 
covered in detail in this specification."

Aka, ebuild's should be written to assume the files they install get 
wiped; there is *zero* mention of mtime, nor could any ebuild rely on 
it and be compliant.

Background as to why we ever relied on mtime- it was a hack to work 
around a bad implementation in portage (treewalk function); it didn't 
actually know if it was replacing or what not, so mtime was what was 
relied on- afaik, that being the sole reason we shoved mtime into 
the vdb also.

At least from the portage standpoint, shifting away from mtime 
reliance was on the radar since '05 and implemented at least 
initially by '06... exact date it was released from a stable branch I 
couldn't tell you, but it's been there a long while.

~brian

Re: [gentoo-dev] Re: POSIX capability in Gentoo

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 499 bytes --]

On Wed, 3 Aug 2011 14:26:56 -0700
Brian Harring <ferringb@gmail.com> wrote:
> Aka, ebuild's should be written to assume the files they install get 
> wiped; there is *zero* mention of mtime, nor could any ebuild rely on 
> it and be compliant.

But as it's a FEATURE, they can't assume that at all.

So either we spec VDB and the unmerge process, which gets horrible for
all kinds of reasons, or ebuilds can't assume that things that have
been modified get wiped.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Re: POSIX capability in Gentoo

[Brian Harring]

On Wed, Aug 03, 2011 at 10:28:51PM +0100, Ciaran McCreesh wrote:
> On Wed, 3 Aug 2011 14:26:56 -0700
> Brian Harring <ferringb@gmail.com> wrote:
> > Aka, ebuild's should be written to assume the files they install get 
> > wiped; there is *zero* mention of mtime, nor could any ebuild rely on 
> > it and be compliant.
> 
> But as it's a FEATURE, they can't assume that at all.

It's outside the ebuild's area of concern (think seperation of 
concerns), just the same as INSTALL_MASK.  The ebuild, per spec, 
should be written to assume it's wiped.

If the user overrides portages make.globals setting FEATURES=unmerge-orphans 
it is on the *users* head to maintain the fallout, just the same as if 
they go and set INSTALL_MASK to do something special.


> So either we spec VDB and the unmerge process, which gets horrible for
> all kinds of reasons, or ebuilds can't assume that things that have
> been modified get wiped.

This is getting more into "the sky is falling" territory.  If you'd 
like to tighten the spec, go nuts, but there isn't anything to see 
here nor is there a real issue.

This really is no different than INSTALL_MASK.
~brian