[gentoo-dev] Requirements for UID/GID management

[gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

We approved GLEP 27 (https://wiki.gentoo.org/wiki/GLEP:27) in 2004 but
never implemented it. I'm wondering what are the explicit requirements
that we have for user and group management?

What I'm really wondering is, instead of the proposal in GLEP27, if we
couldn't simply handle users like any other package. For example,
net-dns/djbdns needs,

  pkg_preinst() {
    # The nofiles group is no longer provided by baselayout.
    # Share it with qmail if possible.
    enewgroup nofiles 200

    enewuser dnscache -1 -1 -1 nofiles
    enewuser dnslog -1 -1 -1 nofiles
    enewuser tinydns -1 -1 -1 nofiles
  }

Instead of that, why couldn't we have something like,

  (R)DEPEND="sys-user/dnscache
             sys-user/dnslog
             sys-user/tinydns"

and then in each of those packages,

  (R)DEPEND="sys-group/nofiles"

That satisfies most of the requirements that *I* have for user and group
management on the system. Compared to the GLEP:

  * EUSERS + EGROUPS: replaced by (R)DEPEND.
  * Defining Accounts: anyone can add a new package already.
  * FEATURES=noautoaccts: use package.provided instead.
  * Local Overrides: use an overlay.
  * users-update: cleanup can be done with --depclean now.

You don't really have to care what UID/GID is assigned, because each
user/group will only be created once and referenced by name (as $PN). By
default, we could pick the first available UID in most packages.
I haven't thought much about the src_install implementation, but it
couldn't be *that* hard. Maybe install a $uid file to /var/lib/portage
somewhere to catch UID conflicts, and keep doing what user.eclass is
doing otherwise.

There isn't a ton of motivation in that GLEP, so I'm not sure what other
use cases I might have overlooked.

Re: [gentoo-dev] Requirements for UID/GID management

[Alexis Ballier]

On Fri, 27 Jan 2017 12:54:07 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:

> That satisfies most of the requirements that *I* have for user and
> group management on the system. Compared to the GLEP:
> 
>   * EUSERS + EGROUPS: replaced by (R)DEPEND.
>   * Defining Accounts: anyone can add a new package already.
>   * FEATURES=noautoaccts: use package.provided instead.
>   * Local Overrides: use an overlay.
>   * users-update: cleanup can be done with --depclean now.

+1

except package.provided which will probably be a mess to handle, but I
dont see why disabling it is a good idea: I'm assuming package will
misbehave at runtime if the user/group is missing

note that i dont think it fixes ROOT!=/ missing users though but
probably paves the way for it

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Fri, Jan 27, 2017 at 12:54 PM, Michael Orlitzky <mjo@gentoo.org> wrote:
>
> You don't really have to care what UID/GID is assigned, because each
> user/group will only be created once and referenced by name (as $PN). By
> default, we could pick the first available UID in most packages.

I might be not following correctly, but due to how filesystems/etc
work it is probably desirable to have consistent UID/GIDs as much as
reasonably possible.  Things like NFS, chroots, containers, and so on
can be a bit simpler if these are consistent, because they involve one
system having visibility into a filesystem hosted on another, and
usually in these cases the UID/GID is what is kept constant, not the
name.  (IMO UID/GID namespace is one of those areas where
Linux/POSIX/etc has some weaknesses.)

This doesn't really seem like a problem though.  Just have a table
somewhere (wiki?) to track who is using what UID/GID and encode those
defaults into the ebuild that creates those users.

Overall I like your proposal.

-- 
Rich

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 01:52 PM, Rich Freeman wrote:
> 
> This doesn't really seem like a problem though.  Just have a table
> somewhere (wiki?) to track who is using what UID/GID and encode those
> defaults into the ebuild that creates those users.
> 

It should be possible to have two different users with the same UID in
the tree, just like we can have two different packages that install the
same file. Eventually somebody will notice a file collision, and then we
add a blocker per usual.

Re: [gentoo-dev] Requirements for UID/GID management

[Gregory Woodbury]

[-- Attachment #1: Type: text/plain, Size: 1644 bytes --]

On Fri, Jan 27, 2017 at 1:52 PM, Rich Freeman <rich0@gentoo.org> wrote:

> On Fri, Jan 27, 2017 at 12:54 PM, Michael Orlitzky <mjo@gentoo.org> wrote:
> >
> > You don't really have to care what UID/GID is assigned, because each
> > user/group will only be created once and referenced by name (as $PN). By
> > default, we could pick the first available UID in most packages.
>
> I might be not following correctly, but due to how filesystems/etc
> work it is probably desirable to have consistent UID/GIDs as much as
> reasonably possible.  Things like NFS, chroots, containers, and so on
> can be a bit simpler if these are consistent, because they involve one
> system having visibility into a filesystem hosted on another, and
> usually in these cases the UID/GID is what is kept constant, not the
> name.  (IMO UID/GID namespace is one of those areas where
> Linux/POSIX/etc has some weaknesses.)
>
> This doesn't really seem like a problem though.  Just have a table
> somewhere (wiki?) to track who is using what UID/GID and encode those
> defaults into the ebuild that creates those users.--
>

There should be a division of the system managed UID space:
1)  constant/consistent UID/GID for major things (portage, etc.)
2)  variable space for per package groups/users that generally don't care
      about consistency

A quick look at /etc/passwd shows that many of the system UIDs are
under 250 (portage) and a few scattered above 400. GIDs are similar,
though some are "fixed" and some are assigned going down from 999.

Some eclasses may need to be scrutinized for what behavior they are using.

-- 
G.Wolfe Woodbury
redwolfe@gmail.com

[-- Attachment #2: Type: text/html, Size: 2413 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Fri, Jan 27, 2017 at 2:35 PM, Michael Orlitzky <mjo@gentoo.org> wrote:
> On 01/27/2017 01:52 PM, Rich Freeman wrote:
>>
>> This doesn't really seem like a problem though.  Just have a table
>> somewhere (wiki?) to track who is using what UID/GID and encode those
>> defaults into the ebuild that creates those users.
>>
>
> It should be possible to have two different users with the same UID in
> the tree, just like we can have two different packages that install the
> same file. Eventually somebody will notice a file collision, and then we
> add a blocker per usual.
>

I'm not saying we can't have random assignment for things where it
doesn't matter, or fall back to random assignment, but it seems rather
silly to go to all the trouble to have blockers when it would be just
as easy to not have a conflict in the first place.  Now, if we have
some longstanding issue from the past that might be another matter,
but what's wrong with just picking an unused ID (again, for stuff that
needs it)?

Telling users that they can't have postfix and apache on the same box
because nobody can be bothered to pick IDs that don't collide seems
like an arbitrary imposition.  Sometimes upstream creates conflicts
that are just hard to work around (same SONAME, different ABI or even
API, and so on).  And if we were talking about some binary-only
upstream package that relies on hardcoded UIDs I guess blockers might
be our only option, and users will just have to be happy that we're
able to support it at all.  However, we shouldn't be the ones creating
these kinds of conflicts.

-- 
Rich

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 02:53 PM, Rich Freeman wrote:
> 
> I'm not saying we can't have random assignment for things where it
> doesn't matter, or fall back to random assignment, but it seems rather
> silly to go to all the trouble to have blockers when it would be just
> as easy to not have a conflict in the first place...
> 
> Telling users that they can't have postfix and apache on the same box
> because nobody can be bothered to pick IDs that don't collide seems
> like an arbitrary imposition.

Agreed, blockers should be a last resort. If neither postfix nor apache
cares about its UID (they don't), then certainly somebody should change
one of them. My first impression is that any package that doesn't care
about its UID should default to "first available", but if that causes
problems, then that's exactly the sort of use case I'm looking for.

If it's a big problem, we can have devs pick an unclaimed UID and stick
with it. Or, if there are 5 users total who need the "apache" user to
have UID 80 across every machine, then it might just make more sense to
tell them to use an overlay to override sys-user/apache.

Keep in mind that currently, if the UID you want isn't available,
user.eclass will shrug and give you a different one. No one can really
rely on consistent UIDs now, but it's not fair to dismiss the idea
because maybe that's one of the things the GLEP was supposed to fix.

Re: [gentoo-dev] Requirements for UID/GID management

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2461 bytes --]

On Fri, 27 Jan 2017 12:54:07 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:

> We approved GLEP 27 (https://wiki.gentoo.org/wiki/GLEP:27) in 2004 but
> never implemented it. I'm wondering what are the explicit requirements
> that we have for user and group management?

I don't think GLEP 27 could be really considered approved if it hasn't
seen any action for 12 years.

> What I'm really wondering is, instead of the proposal in GLEP27, if we
> couldn't simply handle users like any other package. For example,
> net-dns/djbdns needs,
> 
>   pkg_preinst() {
>     # The nofiles group is no longer provided by baselayout.
>     # Share it with qmail if possible.
>     enewgroup nofiles 200
> 
>     enewuser dnscache -1 -1 -1 nofiles
>     enewuser dnslog -1 -1 -1 nofiles
>     enewuser tinydns -1 -1 -1 nofiles
>   }
> 
> Instead of that, why couldn't we have something like,
> 
>   (R)DEPEND="sys-user/dnscache
>              sys-user/dnslog
>              sys-user/tinydns"
> 
> and then in each of those packages,
> 
>   (R)DEPEND="sys-group/nofiles"

At a first glance it seems like a lot of effort for a problem that's
already well-solved on the eclass level. However, it probably makes
sense when you consider users/groups created by multiple packages.

> That satisfies most of the requirements that *I* have for user and group
> management on the system. Compared to the GLEP:
> 
>   * EUSERS + EGROUPS: replaced by (R)DEPEND.
>   * Defining Accounts: anyone can add a new package already.
>   * FEATURES=noautoaccts: use package.provided instead.
>   * Local Overrides: use an overlay.
>   * users-update: cleanup can be done with --depclean now.

Err, cleanup is never easy. You shouldn't really remove a user if it
owns any files. I guess you could abuse pkg_prerm() for that but
depclean will be terribly slow then.

> You don't really have to care what UID/GID is assigned, because each
> user/group will only be created once and referenced by name (as $PN). By
> default, we could pick the first available UID in most packages.
> I haven't thought much about the src_install implementation, but it
> couldn't be *that* hard. Maybe install a $uid file to /var/lib/portage
> somewhere to catch UID conflicts, and keep doing what user.eclass is
> doing otherwise.

Let's invent /etc/passwd.d and /etc/group.d ;-P.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Fri, Jan 27, 2017 at 3:09 PM, Michael Orlitzky <mjo@gentoo.org> wrote:
> My first impression is that any package that doesn't care
> about its UID should default to "first available", but if that causes
> problems, then that's exactly the sort of use case I'm looking for.
>

The ones I listed before were filesystems shared by multiple hosts,
such as with nfs, containers, and chroots.  Granted, there are ways to
deal with this sort of thing, but if you want to share your /var/www
across a bunch of apache servers it would be nice if they all had the
same UID for apache.

-- 
Rich

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 04:15 PM, Michał Górny wrote:
>
>>   * users-update: cleanup can be done with --depclean now.
> 
> Err, cleanup is never easy. You shouldn't really remove a user if it
> owns any files. I guess you could abuse pkg_prerm() for that but
> depclean will be terribly slow then.
> 

What are our options if the user uninstalls a package that had its own
system user, and that system user still owns some files? Here are the
first few that come to my mind:

  1) Leave the user there but uninstall the sys-user/foo package. This
     is basically what we do now.

  2) Bail out of pkg_prerm and tell the user to clean up those files.

  3) Kill off the user anyway and leave the files.

The problem with (1) is that every user on the system is a liability,
and they never go away. If you test-compiled postgresql once five years
ago, there's still a "postgres" user with a shell on your system today.
Is the password locked? If not, is it strong? I have to ask the same
question about every other junk user in my /etc/passwd. There's also no
easy way to figure out which users are still needed at a later point.

The problem with (2) is that it's slow and annoying. Telling the user to
call "chown" on those files isn't safe, but I guess we could output a
"find ... -delete" command to be copy/pasted. We could also arrange
things so that if the user gets rid of the system user/group himself,
then the uninstall would succeed: basically a shut-up-portage override.
It'd still be horribly slow though.

The problem with (3) is that if some new user -- let's say "nobody" --
gets added to the system, it could be assigned a (previously removed)
UID that owns sensitive files. This would be abated by hard-coding UIDs
instead of taking the first available.

Hard-coding UIDs comes with its own set of problems though. It doesn't
totally solve the problem in (3), because users added outside of portage
could still be assigned a UID that owns files. And how do we guarantee
that a hard-coded UID won't be usurped? Some crappy build system could
throw a UID 80 onto the system and the user might never know until it
conflicts with apache. And right now we've got UID_MIN=1000 in
login.defs -- are we sure that we'll never have more than 1,000 sys-user
packages?

Re: [gentoo-dev] Requirements for UID/GID management

[Patrick McLean]

On Fri, 27 Jan 2017 14:53:18 -0500
Rich Freeman <rich0@gentoo.org> wrote:

> On Fri, Jan 27, 2017 at 2:35 PM, Michael Orlitzky <mjo@gentoo.org>
> wrote:
> > On 01/27/2017 01:52 PM, Rich Freeman wrote:  
> >>
> >> This doesn't really seem like a problem though.  Just have a table
> >> somewhere (wiki?) to track who is using what UID/GID and encode
> >> those defaults into the ebuild that creates those users.
> >>  
> >
> > It should be possible to have two different users with the same UID
> > in the tree, just like we can have two different packages that
> > install the same file. Eventually somebody will notice a file
> > collision, and then we add a blocker per usual.
> >  
> 
> I'm not saying we can't have random assignment for things where it
> doesn't matter, or fall back to random assignment, but it seems rather
> silly to go to all the trouble to have blockers when it would be just
> as easy to not have a conflict in the first place.  Now, if we have
> some longstanding issue from the past that might be another matter,
> but what's wrong with just picking an unused ID (again, for stuff that
> needs it)?
> 
> Telling users that they can't have postfix and apache on the same box
> because nobody can be bothered to pick IDs that don't collide seems
> like an arbitrary imposition.  Sometimes upstream creates conflicts
> that are just hard to work around (same SONAME, different ABI or even
> API, and so on).  And if we were talking about some binary-only
> upstream package that relies on hardcoded UIDs I guess blockers might
> be our only option, and users will just have to be happy that we're
> able to support it at all.  However, we shouldn't be the ones creating
> these kinds of conflicts.
> 

I don't think we need to have stable UIDs/GIDs in the "normal" case of
standalone users with a single Gentoo system at home. The people who
need predictable UIDs/GIDs are the "enterprise" users or the home users
who use things such as NFS. I work for a company that uses Gentoo, we
have a bunch of workarounds to make sure that UIDs and GIDs are
stable. To make something to solve our problem (and I suspect everyone
else who cares about this), it would be sufficient to have a mechanism
to override the default random assignment with a fixed UID/GID.
Possibly some file in /etc/portage or in the profile (or both) that
allows one to configure what UID/GID a user will get when the user is
being created. One advantage of this is that user.eclass could be
modified to support it, so we don't have to wait for a new EAPI before
taking advantage of it.

[gentoo-dev] Re: Requirements for UID/GID management

[Duncan]

Rich Freeman posted on Fri, 27 Jan 2017 16:23:02 -0500 as excerpted:

> On Fri, Jan 27, 2017 at 3:09 PM, Michael Orlitzky <mjo@gentoo.org>
> wrote:
>> My first impression is that any package that doesn't care about its UID
>> should default to "first available", but if that causes problems, then
>> that's exactly the sort of use case I'm looking for.
>>
>>
> The ones I listed before were filesystems shared by multiple hosts,
> such as with nfs, containers, and chroots.  Granted, there are ways to
> deal with this sort of thing, but if you want to share your /var/www
> across a bunch of apache servers it would be nice if they all had the
> same UID for apache.

That's what an admin should be taking care of... if they have reason 
(like the given multiple machines accessing the same filesystem reason) 
to care.

And the way they'd do it under this proposal is simple enough.  Simply 
stick the admin-uid/apache (or whatever) ebuild in their overlay, 
uncomment the line in it that sets a specific UID instead of picking the 
next one in sequence, and change that specific UID if necessary for their 
installation.

The admin-uid/* ebuilds, meanwhile, could be pretty much empty save for 
two assignment lines, the commented specific UID assignment and an 
uncommented one listing the user name, and an eclass inherit, with the 
eclass simply reading the assigned name and picking a UID randomly if 
it's not already assigned, either by the user uncommenting the assignment 
line in the ebuild in their overlay, or a previous installation.

Of course the eclass could also check for an override variable, which 
would allow the user a second way of specifying UIDs -- via package.env 
or the like, similar to the way git-r3 allows environmental override of 
commit, etc.

(I say UID above, GID would be handled similarly, presumably in the same 
ebuilds and eclass, with different vars, of course.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 09:37 PM, Patrick McLean wrote:
> 
> To make something to solve our problem (and I suspect everyone
> else who cares about this), it would be sufficient to have a mechanism
> to override the default random assignment with a fixed UID/GID.

What I had in mind for this is that a "normal" user ebuild would look
like...

  EAPI=6
  # SYS_USER_NAME defaults to ${PN}
  SYS_USER_GROUPS="foo" # defaults empty or to ${PN}, not sure
  # SYS_USER_UID left undefined to get the next available
  # SYS_USER_HOME left undefined to get the default
  # SYS_USER_SHELL left undefined... etc.
  inherit sys-user.eclass
  ...
  # DEPEND set to "sys-group/foo" by the eclass
  # RDEPEND set to "sys-group/foo" by the eclass
  ...
  # src_install and pkg_prerm from the eclass.

To override the UID behavior, I think package.env would work, but an
overlay where the ebuild has SYS_USER_UID=42 is guaranteed to.

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Fri, Jan 27, 2017 at 9:37 PM, Patrick McLean <chutzpah@gentoo.org> wrote:
>
> I don't think we need to have stable UIDs/GIDs in the "normal" case of
> standalone users with a single Gentoo system at home.

Of course, but as you point out the enterprise case has more
sophisticated solutions.  I think the case of somebody running in a
home setting or a very small server setting is the main issue.

It isn't like inconsistent UIDs are the end of the world.  However,
IMO it still makes sense to at least try to standardize such things.
Really, if you have a package always installing the same user simply
sticking a default UID without any effort to avoid collisions is
better than nothing, but having a wiki page where people can register
UIDs isn't that big a deal.

If it fails, it fails...

-- 
Rich

Re: [gentoo-dev] Requirements for UID/GID management

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 1713 bytes --]

On Fri, 27 Jan 2017 18:37:52 -0800
Patrick McLean <chutzpah@gentoo.org> wrote:

> I don't think we need to have stable UIDs/GIDs in the "normal" case of
> standalone users with a single Gentoo system at home. The people who
> need predictable UIDs/GIDs are the "enterprise" users or the home users
> who use things such as NFS. I work for a company that uses Gentoo, we
> have a bunch of workarounds to make sure that UIDs and GIDs are
> stable. To make something to solve our problem (and I suspect everyone
> else who cares about this), it would be sufficient to have a mechanism
> to override the default random assignment with a fixed UID/GID.
> Possibly some file in /etc/portage or in the profile (or both) that
> allows one to configure what UID/GID a user will get when the user is
> being created. One advantage of this is that user.eclass could be
> modified to support it, so we don't have to wait for a new EAPI before
> taking advantage of it.

Is this really a problem in enterprise? What are the workarounds you're
using? NFS has long had idmapd, which takes care of this problem. I
still find people shy away from NFSv4 but I've not had any trouble with
it. There's also LDAP, usually coupled with sssd these days, in which
case the users and groups are created just once on a central server.
Samba with Active Directory effectively gives you the same thing and
can also be coupled with sssd. I recently tried mixing Samba, sssd, and
NFS, which was quite fascinating and surprisingly easy thanks to
realmd. This allowed me to use NFS with Kerberos, which is something
you really need in an enterprise environment.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Tom H]

On Fri, Jan 27, 2017 at 1:52 PM, Rich Freeman <rich0@gentoo.org> wrote:
>
> Just have a table somewhere (wiki?) to track who is using what UID/GID
> and encode those defaults into the ebuild that creates those users.

FYI, how Debian and Fedora do it:

Debian policy
https://www.debian.org/doc/debian-policy/ch-opersys.html#s9.2.2

Debian users
https://anonscm.debian.org/cgit/users/cjwatson/base-passwd.git/plain/passwd.master

Debian groups
https://anonscm.debian.org/cgit/users/cjwatson/base-passwd.git/plain/group.master

Fedora users and groups
https://git.fedorahosted.org/cgit/setup.git/plain/uidgid

Re: [gentoo-dev] Requirements for UID/GID management

[A. Wilcox]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 27/01/17 20:37, Patrick McLean wrote:
> I don't think we need to have stable UIDs/GIDs in the "normal" case
> of standalone users with a single Gentoo system at home. The people
> who need predictable UIDs/GIDs are the "enterprise" users or the
> home users who use things such as NFS. I work for a company that
> uses Gentoo, we have a bunch of workarounds to make sure that UIDs
> and GIDs are stable. To make something to solve our problem (and I
> suspect everyone else who cares about this), it would be sufficient
> to have a mechanism to override the default random assignment with
> a fixed UID/GID. Possibly some file in /etc/portage or in the
> profile (or both) that allows one to configure what UID/GID a user
> will get when the user is being created. One advantage of this is
> that user.eclass could be modified to support it, so we don't have
> to wait for a new EAPI before taking advantage of it.
> 


Having a file that user.eclass would use to map new users/groups to
IDs would be extremely beneficial to me.  I was thinking about diving
in to that some time later, after the GLEP 70 work I'm doing, but if
someone else wants to take it - please!  That would greatly ease the
pain of not only NFS, but swapping data disks around between different /
.

Consider, for example, one of my use cases for this:  I have a
LibreSSL / that I use solely for testing ebuilds against it, and my
regular / with OpenSSL.  I share /home and /srv between these two, but
the apache, nginx, and charybdis users have different UIDs between
them.  Therefore I have to chown -R each time I test LibreSSL.

I could use a different /home and /srv, or make two copies, but it's
much easier for me to test these apps having my entire normal
environment available to me.

Best,
- --arw


- -- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJYjN9aAAoJEMspy1GSK50UD3YQAI/ImKKEoTdEu9B3woyDsbcz
4QElt5OsaiOFcT9f30/rI/8G5NQ9JYbh/XvvS7JlPllhCu+xV+BQhGznH7C+w7sf
0m/9HJbJLLDXcpP1gB9lUTn1JhKN6Vp19UhTi5upXIhTK5yTeUAxG/VhpucfMnml
QsC7DOg584nL47/w2jc/IWqZLIJ/SVbWcYBpjbfelCRHetuR/cXLdpe4EhqnwcVx
EhVh1zUJYDMDwFK5OYCrwHFvp2PUy7d1qiWOJZ5dGvw+SuG2/Xd2hcwgwFf6X3EK
8cxWPc3xrbmmtxKTatkKB/pOGn1rj/bm4JD0XxjzPJUWJ28eZ06LDZ2lszm7xse5
KUg0cgT4AwER0K1G7bqFfntdDNii6qjs/B5oBY9Jr/SC0YGDvcbh2bMYKDRTDRqN
Qu9zzk5MndkoIOQUFt5ccRYoXftDBKofmqWYhqjxo/LUcnvpF9w1nacsGIkkFWE6
64Y80yIr1A++WQGasd2U1SAbDFFHaXdv5YSENRTGo19I/QWVO1L70M3KRh7YIgz/
Nx7aWH3ir9BGFqi/plqSfbr30m85EA1LMnc8iPfe/HcnyOZgrZkdNzmENoeudqGU
SAe5AeAimoYbmJfJAv4ou4aOnKtNi4tZQVTkYi4Y9SvkRLHU7lquPDre92f9WWOx
jMHKVwcPi3BGtladWZM1
=Ufwn
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Requirements for UID/GID management

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 1138 bytes --]

On Sat, 28 Jan 2017 12:13:53 -0600
"A. Wilcox" <awilfox@adelielinux.org> wrote:

> Having a file that user.eclass would use to map new users/groups to
> IDs would be extremely beneficial to me.  I was thinking about diving
> in to that some time later, after the GLEP 70 work I'm doing, but if
> someone else wants to take it - please!  That would greatly ease the
> pain of not only NFS, but swapping data disks around between different /
> .
> 
> Consider, for example, one of my use cases for this:  I have a
> LibreSSL / that I use solely for testing ebuilds against it, and my
> regular / with OpenSSL.  I share /home and /srv between these two, but
> the apache, nginx, and charybdis users have different UIDs between
> them.  Therefore I have to chown -R each time I test LibreSSL.
> 
> I could use a different /home and /srv, or make two copies, but it's
> much easier for me to test these apps having my entire normal
> environment available to me.

As mentioned in my other post, why are you not using idmapd? It's
trivial to set up on top of NFSv4.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Sat, Jan 28, 2017 at 2:32 PM, James Le Cuirot <chewi@gentoo.org> wrote:
> On Sat, 28 Jan 2017 12:13:53 -0600
> "A. Wilcox" <awilfox@adelielinux.org> wrote:
>
>> Having a file that user.eclass would use to map new users/groups to
>> IDs would be extremely beneficial to me.  I was thinking about diving
>> in to that some time later, after the GLEP 70 work I'm doing, but if
>> someone else wants to take it - please!  That would greatly ease the
>> pain of not only NFS, but swapping data disks around between different /
>> .
>>
>> Consider, for example, one of my use cases for this:  I have a
>> LibreSSL / that I use solely for testing ebuilds against it, and my
>> regular / with OpenSSL.  I share /home and /srv between these two, but
>> the apache, nginx, and charybdis users have different UIDs between
>> them.  Therefore I have to chown -R each time I test LibreSSL.
>>
>> I could use a different /home and /srv, or make two copies, but it's
>> much easier for me to test these apps having my entire normal
>> environment available to me.
>
> As mentioned in my other post, why are you not using idmapd? It's
> trivial to set up on top of NFSv4.

As far as I can tell there is no Gentoo-specific documentation for
doing this, and from what I have read setting up NFSv4 is a PITA
(perhaps that has changed in recent years).  There are also use cases
that don't involve NFS, such as containers.  From the docs I have
found on idmapd there wasn't actually a lot of detail, it wasn't clear
if it "just works" without any specific configuration, perhaps it
does.

In any case, would it be that hard to set reasonable defaults?

-- 
Rich

Re: [gentoo-dev] Requirements for UID/GID management

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 2762 bytes --]

On Sat, 28 Jan 2017 15:34:01 -0500
Rich Freeman <rich0@gentoo.org> wrote:

> On Sat, Jan 28, 2017 at 2:32 PM, James Le Cuirot <chewi@gentoo.org> wrote:
> > On Sat, 28 Jan 2017 12:13:53 -0600
> > "A. Wilcox" <awilfox@adelielinux.org> wrote:
> >  
> >> Having a file that user.eclass would use to map new users/groups to
> >> IDs would be extremely beneficial to me.  I was thinking about diving
> >> in to that some time later, after the GLEP 70 work I'm doing, but if
> >> someone else wants to take it - please!  That would greatly ease the
> >> pain of not only NFS, but swapping data disks around between different /
> >> .
> >>
> >> Consider, for example, one of my use cases for this:  I have a
> >> LibreSSL / that I use solely for testing ebuilds against it, and my
> >> regular / with OpenSSL.  I share /home and /srv between these two, but
> >> the apache, nginx, and charybdis users have different UIDs between
> >> them.  Therefore I have to chown -R each time I test LibreSSL.
> >>
> >> I could use a different /home and /srv, or make two copies, but it's
> >> much easier for me to test these apps having my entire normal
> >> environment available to me.  
> >
> > As mentioned in my other post, why are you not using idmapd? It's
> > trivial to set up on top of NFSv4.  
> 
> As far as I can tell there is no Gentoo-specific documentation for
> doing this, and from what I have read setting up NFSv4 is a PITA
> (perhaps that has changed in recent years).  There are also use cases
> that don't involve NFS, such as containers.  From the docs I have
> found on idmapd there wasn't actually a lot of detail, it wasn't clear
> if it "just works" without any specific configuration, perhaps it
> does.

The only common complaints I can recall about NFSv4 over v3 are having
to export everything under a single root directory and needing to set
fsid against each export for some inexplicable reason. Just bind mount
everything you want to export and set a different fsid number against
each export line. It's odd when you're used to v3 but really not that
hard. I've been using it for years and found it to be faster and more
reliable.

As for idmapd, this is my entire configuration file. I don't even think
the last section is necessary. Configure and start it on every node. It
really does just work.

[General]
Domain = aura-online.co.uk

[Mapping]
Nobody-User = nobody
Nobody-Group = nobody

[Translation]
Method = nsswitch

> In any case, would it be that hard to set reasonable defaults?

I do think this idea is a good one. I just wasn't buying the enterprise
argument and was surprised that no one had even mentioned idmapd.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Patrick McLean]

On Sat, 28 Jan 2017 11:28:45 +0000
James Le Cuirot <chewi@gentoo.org> wrote:

> On Fri, 27 Jan 2017 18:37:52 -0800
> Patrick McLean <chutzpah@gentoo.org> wrote:
> 
> > I don't think we need to have stable UIDs/GIDs in the "normal" case of
> > standalone users with a single Gentoo system at home. The people who
> > need predictable UIDs/GIDs are the "enterprise" users or the home users
> > who use things such as NFS. I work for a company that uses Gentoo, we
> > have a bunch of workarounds to make sure that UIDs and GIDs are
> > stable. To make something to solve our problem (and I suspect everyone
> > else who cares about this), it would be sufficient to have a mechanism
> > to override the default random assignment with a fixed UID/GID.
> > Possibly some file in /etc/portage or in the profile (or both) that
> > allows one to configure what UID/GID a user will get when the user is
> > being created. One advantage of this is that user.eclass could be
> > modified to support it, so we don't have to wait for a new EAPI before
> > taking advantage of it.  
> 
> Is this really a problem in enterprise? What are the workarounds you're
> using? NFS has long had idmapd, which takes care of this problem. I
> still find people shy away from NFSv4 but I've not had any trouble with
> it. There's also LDAP, usually coupled with sssd these days, in which
> case the users and groups are created just once on a central server.
> Samba with Active Directory effectively gives you the same thing and
> can also be coupled with sssd. I recently tried mixing Samba, sssd, and
> NFS, which was quite fascinating and surprisingly easy thanks to
> realmd. This allowed me to use NFS with Kerberos, which is something
> you really need in an enterprise environment.
> 

We are using both NFSv3 and NFSv4, the UID stability is also important
when you are using full-image deployments, and have local storage on
the system, you don't want the new OS to have different UIDs/GIDs than
the previous installation.

We are using file in /etc/portage/env that define a pre_pkg_setup that
creates the users before the standard pkg_setup does, with our stable
UID/GID for that system. We have to do this for each package that
creates a user that may be used to store stable data.

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 11:21 PM, Rich Freeman wrote:
> 
> It isn't like inconsistent UIDs are the end of the world.  However,
> IMO it still makes sense to at least try to standardize such things.
> Really, if you have a package always installing the same user simply
> sticking a default UID without any effort to avoid collisions is
> better than nothing, but having a wiki page where people can register
> UIDs isn't that big a deal.
> 

I threw together an ugly implementation so I could play with both
approaches -- random or fixed UIDs by default. The code to get user and
group management working is of course nice and simple in either case.
Where they both turn to shit is the upgrade path.

Here's a problem I have no solution for. Suppose we tell everyone to
pick a fixed UID for their user packages. I have a randomly assigned
"tcpdump" user as UID 102 on my machine today. If we roll this out next
week and the tcpdump maintainer chooses UID=321 as his fixed UID, what
happens when I go to install sys-user/tcpdump? Every option is bad:

  * Keep the existing user. Now its UID is wrong. You might say "so
    what," but the majority of users on the majority of systems are
    going to have this problem, so you have to wonder what we've
    gained by deciding on fixed UIDs and then ultimately assigning
    them randomly anyway.

    There's the related problem of what to do if the tuxracer maintainer
    decides he wants to use UID=102 and I still have tcpdump using it.

  * Overwrite the existing user with the new one. Your packages all
    break.

  * Have the ebuild die(), and tell the user to fix the UID and file
    ownership himself before emerge can continue. Good luck with that.

In the mostly-random-UIDs approach, I have an answer, even if it's not
pretty: I can use the pre-existing UID instead of the next available
one. This still fails if the ebuild author requests a specific
(conflicting) UID, but that should be extremely rare in the random-UIDs
model.

Can anyone think of an upgrade path for fixed UIDs? That issue aside, I
may have convinced myself that fixed UIDs are better.

Re: [gentoo-dev] Requirements for UID/GID management

[Rich Freeman]

On Sat, Jan 28, 2017 at 8:56 PM, Michael Orlitzky <mjo@gentoo.org> wrote:
> On 01/27/2017 11:21 PM, Rich Freeman wrote:
>>
>> It isn't like inconsistent UIDs are the end of the world.  However,
>> IMO it still makes sense to at least try to standardize such things.
>> Really, if you have a package always installing the same user simply
>> sticking a default UID without any effort to avoid collisions is
>> better than nothing, but having a wiki page where people can register
>> UIDs isn't that big a deal.
>>
>
> Here's a problem I have no solution for. Suppose we tell everyone to
> pick a fixed UID for their user packages. I have a randomly assigned
> "tcpdump" user as UID 102 on my machine today. If we roll this out next
> week and the tcpdump maintainer chooses UID=321 as his fixed UID, what
> happens when I go to install sys-user/tcpdump? Every option is bad:
>
>   * Keep the existing user. Now its UID is wrong. You might say "so
>     what," but the majority of users on the majority of systems are
>     going to have this problem, so you have to wonder what we've
>     gained by deciding on fixed UIDs and then ultimately assigning
>     them randomly anyway.

Honestly, I really will say "so what" here.  :)

Sure, it isn't a perfect solution, but it costs you nothing, and the
fallback is just random UIDs, which as we've already established
aren't a huge problem.  For new installs things will be more
consistent.

It is of course possible to remap UIDs, but I don't think we should
ever try to do this automatically, because only the user can know if
every filesystem that might contain the old UIDs is actually mounted,
or if they mind find killing their drives at the moment, or if
anything important is running under the old uid.

I'm sure somebody will end up offering up a script at some point that
will remap an existing Gentoo install in single user mode to the new
defaults if somebody wishes to do so.

The bottom line is that I think at least picking some defaults is
going to result in a typical new install having matching uids, which
is going to make life easier for small-scale multi-host setups (NFS,
containers, etc).  No, it will never work at the enterprise scale (for
starters, other distros will probably come into play), and it doesn't
matter for a standalone box.  However, just putting a stick in the mud
will give 95% of the benefit for zero additional work.  And the
fallback to random IDs is already implemented anyway.

So, don't try to fix the decades-old boxes.  By now everybody who has
them has beards gray enough to deal with any issues, and they'll have
to have been dealing with them all along anyway.

-- 
Rich

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/28/2017 09:22 PM, Rich Freeman wrote:
>>
>> Here's a problem I have no solution for. Suppose we tell everyone to
>> pick a fixed UID for their user packages. I have a randomly assigned
>> "tcpdump" user as UID 102 on my machine today. If we roll this out next
>> week and the tcpdump maintainer chooses UID=321 as his fixed UID, what
>> happens when I go to install sys-user/tcpdump? Every option is bad:
>>
>>   * Keep the existing user. Now its UID is wrong. You might say "so
>>     what," but the majority of users on the majority of systems are
>>     going to have this problem, so you have to wonder what we've
>>     gained by deciding on fixed UIDs and then ultimately assigning
>>     them randomly anyway.
> 
> Honestly, I really will say "so what" here.  :)
> 
> Sure, it isn't a perfect solution, but it costs you nothing, and the
> fallback is just random UIDs, which as we've already established
> aren't a huge problem.  For new installs things will be more
> consistent.
> 


"Fixed UIDs with random fallback" and "random UIDs" are the same thing.
If our goal is to make things easy on people who need fixed UIDs, then
they should be truly fixed. Even on a new machine, you might wind up
with all of the preferred UIDs, or they might all be wrong. Suppose I
add a new user 21 on my fresh install. The ftp server was going to use
21, but now it falls back to 22. The SSH daemon was going to take 22,
but now it falls back to 23. The telnet daemon was going to take 23...
That doesn't help NFS users. It's still random in the sense that it
depends on the state of the system in an unpredictable way.

And what about packages that truly require a fixed UID? I think that's
the only place my current implementation and your proposal differ. I
have it die() under the assumption that most ebuilds won't request a
UID. If everyone picks a UID and we can ignore it, what do we do about
the packages that actually need what they asked for?

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/28/2017 09:22 PM, Rich Freeman wrote:
> 
> Honestly, I really will say "so what" here.  :)
> 

I forgot to mention a few of the advantages of having really-fixed UIDs.
First, it makes the code simpler. Yup, cool.

It also lets us play a nice trick and use the UID as a subslot, so that
if some sys-user/foo package ever changes its UID, everything depending
on it can be rebuilt to use the new UID.

Finally, it makes the installed image deterministic. Right now to
prevent file collisions, we have to read stuff from /etc/passwd in order
to generate a file that gets installed to /var/lib. The installed image
of a package shouldn't change on-the-fly like that, and it wouldn't if
the UID was truly fixed.

We don't get any of those benefits if we're falling back to random UIDs.

Re: [gentoo-dev] Requirements for UID/GID management

[M. J. Everitt]

[-- Attachment #1.1: Type: text/plain, Size: 2537 bytes --]

On 29/01/17 01:56, Michael Orlitzky wrote:
> On 01/27/2017 11:21 PM, Rich Freeman wrote:
>> It isn't like inconsistent UIDs are the end of the world.  However,
>> IMO it still makes sense to at least try to standardize such things.
>> Really, if you have a package always installing the same user simply
>> sticking a default UID without any effort to avoid collisions is
>> better than nothing, but having a wiki page where people can register
>> UIDs isn't that big a deal.
>>
> I threw together an ugly implementation so I could play with both
> approaches -- random or fixed UIDs by default. The code to get user and
> group management working is of course nice and simple in either case.
> Where they both turn to shit is the upgrade path.
>
> Here's a problem I have no solution for. Suppose we tell everyone to
> pick a fixed UID for their user packages. I have a randomly assigned
> "tcpdump" user as UID 102 on my machine today. If we roll this out next
> week and the tcpdump maintainer chooses UID=321 as his fixed UID, what
> happens when I go to install sys-user/tcpdump? Every option is bad:
>
>   * Keep the existing user. Now its UID is wrong. You might say "so
>     what," but the majority of users on the majority of systems are
>     going to have this problem, so you have to wonder what we've
>     gained by deciding on fixed UIDs and then ultimately assigning
>     them randomly anyway.
>
>     There's the related problem of what to do if the tuxracer maintainer
>     decides he wants to use UID=102 and I still have tcpdump using it.
>
>   * Overwrite the existing user with the new one. Your packages all
>     break.
>
>   * Have the ebuild die(), and tell the user to fix the UID and file
>     ownership himself before emerge can continue. Good luck with that.
>
> In the mostly-random-UIDs approach, I have an answer, even if it's not
> pretty: I can use the pre-existing UID instead of the next available
> one. This still fails if the ebuild author requests a specific
> (conflicting) UID, but that should be extremely rare in the random-UIDs
> model.
>
> Can anyone think of an upgrade path for fixed UIDs? That issue aside, I
> may have convinced myself that fixed UIDs are better.
>
>
I suspect that this would be best handled in an eclass with a stack of
functions controlled by USE-flags or an eselect module or something.
Seriously outta my league on this one .. but throwing the idea out there
for anyone to discuss potential viability or not!


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Gordon Pettey]

[-- Attachment #1: Type: text/plain, Size: 625 bytes --]

On Sat, Jan 28, 2017 at 8:54 PM, Michael Orlitzky <mjo@gentoo.org> wrote:

> On 01/28/2017 09:22 PM, Rich Freeman wrote:
> >
> > Honestly, I really will say "so what" here.  :)
> >
>
> I forgot to mention a few of the advantages of having really-fixed UIDs.
> First, it makes the code simpler. Yup, cool.
>
> It also lets us play a nice trick and use the UID as a subslot, so that
> if some sys-user/foo package ever changes its UID, everything depending
> on it can be rebuilt to use the new UID.
>

That's nonsense for reasons already mentioned by rich0. UIDs don't change
except in the case of an admin doing it manually.

[-- Attachment #2: Type: text/html, Size: 1039 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[M. J. Everitt]

[-- Attachment #1.1.1: Type: text/plain, Size: 1211 bytes --]

On 29/01/17 03:23, Gordon Pettey wrote:
> On Sat, Jan 28, 2017 at 8:54 PM, Michael Orlitzky <mjo@gentoo.org
> <mailto:mjo@gentoo.org>> wrote:
>
>     On 01/28/2017 09:22 PM, Rich Freeman wrote:
>     >
>     > Honestly, I really will say "so what" here.  :)
>     >
>
>     I forgot to mention a few of the advantages of having really-fixed
>     UIDs.
>     First, it makes the code simpler. Yup, cool.
>
>     It also lets us play a nice trick and use the UID as a subslot, so
>     that
>     if some sys-user/foo package ever changes its UID, everything
>     depending
>     on it can be rebuilt to use the new UID.
>
>
> That's nonsense for reasons already mentioned by rich0. UIDs don't
> change except in the case of an admin doing it manually.
The question is not whether the UIDs have or are changing .. its whether
they were the same in the first place. With the exception possibly of
two+ identical installs (ie. same base image, same portage snapshot,
same install sequence, same USE flags, etc etc etc) the case of a
non-identical setup may cause slight differences (a common "off-by-one"
error) in UID allocation if "next available" is the default allocation
option.

[-- Attachment #1.1.2: Type: text/html, Size: 2320 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/28/2017 10:23 PM, Gordon Pettey wrote:
> 
> That's nonsense for reasons already mentioned by rich0. UIDs don't change
> except in the case of an admin doing it manually.
> 

It shouldn't be common, but it can and will happen once you put users in
ebuilds. As an example, imagine an "echo" daemon that listens on a port
and echoes every line of text sent to it. When building the daemon, you
compile it --with-uid=<whatever>, and it runs as that restricted user
with access to nothing. In the ebuild, you depend on sys-user/echo, and
compile the daemon with the current UID for that user.

Now a year later, someone makes a new revision of sys-user/echo and
gives it a new fixed UID. Oops, your daemon broke. How do you fix it?
Rebuild it. You want to rebuild the daemon whenever the UID in the
sys-user/echo package changes. That's exactly what subslot deps do.

Disregard whether or not it's stupid to do, or whether it will work in
all cases. *If* a system user *does* change its UID, then you almost
certainly want to rebuild the packages that depend on it. That can be
done if everything has a fixed UID determined by a variable in the
ebuild, and the new user is guaranteed to get that UID.

Re: [gentoo-dev] Requirements for UID/GID management

[Alan McKinnon]

On 29/01/2017 03:56, Michael Orlitzky wrote:
> On 01/27/2017 11:21 PM, Rich Freeman wrote:
>>
>> It isn't like inconsistent UIDs are the end of the world.  However,
>> IMO it still makes sense to at least try to standardize such things.
>> Really, if you have a package always installing the same user simply
>> sticking a default UID without any effort to avoid collisions is
>> better than nothing, but having a wiki page where people can register
>> UIDs isn't that big a deal.
>>
> 
> I threw together an ugly implementation so I could play with both
> approaches -- random or fixed UIDs by default. The code to get user and
> group management working is of course nice and simple in either case.
> Where they both turn to shit is the upgrade path.
> 
> Here's a problem I have no solution for. Suppose we tell everyone to
> pick a fixed UID for their user packages. I have a randomly assigned
> "tcpdump" user as UID 102 on my machine today. If we roll this out next
> week and the tcpdump maintainer chooses UID=321 as his fixed UID, what
> happens when I go to install sys-user/tcpdump? Every option is bad:
> 
>   * Keep the existing user. Now its UID is wrong. You might say "so
>     what," but the majority of users on the majority of systems are
>     going to have this problem, so you have to wonder what we've
>     gained by deciding on fixed UIDs and then ultimately assigning
>     them randomly anyway.
> 
>     There's the related problem of what to do if the tuxracer maintainer
>     decides he wants to use UID=102 and I still have tcpdump using it.
> 
>   * Overwrite the existing user with the new one. Your packages all
>     break.
> 
>   * Have the ebuild die(), and tell the user to fix the UID and file
>     ownership himself before emerge can continue. Good luck with that.
> 
> In the mostly-random-UIDs approach, I have an answer, even if it's not
> pretty: I can use the pre-existing UID instead of the next available
> one. This still fails if the ebuild author requests a specific
> (conflicting) UID, but that should be extremely rare in the random-UIDs
> model.
> 
> Can anyone think of an upgrade path for fixed UIDs? That issue aside, I
> may have convinced myself that fixed UIDs are better.

The general process I would recommend is that if the ebuild finds the user
already exists, leave it, it's UID and it's file ownerships alone, and keep
them as they are. If the user does not exist then create it. Preferably
use a pre-assigned UID/GID so there is some consistency with most other
Gentoo things out there.

If the user already exists, it's presumably because the sysadmin wants
it that way or it was installed that way from an older ebuild. Either
way the ebuild cannot mess around with that. It could output an elog
saying the uid/gid doesn't match the new Gentoo norm, and provide the
commands to run to bring things into line (usermod, groupmod, find /
-user -exec chown, etc, etc)



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] Requirements for UID/GID management

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 454 bytes --]

>>>>> On Sat, 28 Jan 2017, Michael Orlitzky wrote:

> [...] sys-user/echo [...]

[Replying to a random message in this thread, as I have some backlog.]

Users and groups aren't packages, so IMHO packages and *DEPEND
variables shouldn't be abused for things like that. This has been
discussed in bug 53269 previously.

IMHO the way to go is to add proper variables in the next EAPI, like
the EUSERS and EGROUPS proposed (and approved) in GLEP 27.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]

On Sun, 29 Jan 2017 11:03:25 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:

> >>>>> On Sat, 28 Jan 2017, Michael Orlitzky wrote:  
> 
> > [...] sys-user/echo [...]  
> 
> [Replying to a random message in this thread, as I have some backlog.]
> 
> Users and groups aren't packages, so IMHO packages and *DEPEND
> variables shouldn't be abused for things like that. This has been
> discussed in bug 53269 previously.
> 
> IMHO the way to go is to add proper variables in the next EAPI, like
> the EUSERS and EGROUPS proposed (and approved) in GLEP 27.

Don't forget to introduce a properly convoluted way of expressing
conditionals on USE flags etc. ;-)

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/29/2017 03:26 AM, Alan McKinnon wrote:
>>
>> Can anyone think of an upgrade path for fixed UIDs? That issue aside, I
>> may have convinced myself that fixed UIDs are better.
> 
> The general process I would recommend is that if the ebuild finds the user
> already exists, leave it, it's UID and it's file ownerships alone, and keep
> them as they are. If the user does not exist then create it.

That's what I've got it doing now...


> Preferably use a pre-assigned UID/GID so there is some consistency
> with most other Gentoo things out there.

This is the only point we have left to consider. To recap, there are
three approaches to try:

  1 Truly fixed IDs. Every user gets the UID it wants, or it doesn't
    get created. The UIDs are all determined beforehand.

    Pros:

      * Cleaner code
      * UIDs agree on all systems
      * Sweet subslot rebuilds
      * Could decide on UIDs to match e.g. Debian

    Cons:

      * Absolutely no way to introduce it into the tree
        without breaking every existing system.


  2 Mostly random UIDs, and the few packages that need to specify one
    can do so. Usually installation will never fail, but if some user
    specifies a particular UID and doesn't get it, we die().

    Pros (w.r.t. option 3):

      * The upgrade path is a little smoother. Most packages don't care
        about their UID, and the UIDs on your system *right now* are all
        random, so the upgrade path is "leave them there."

    Cons (w.r.t. option 3):

      * The UIDs on your system will never be predictable, and anyone
        who needs their UIDs consistent across systems will have to
        create an overlay that changes the behavior of a few ebuilds.

  3 Mostly fixed UIDs, but with a fallback to random ones if you don't
    get the UID you want. Here, everyone specifies their "preferred"
    UID, and we try that first. If it doesn't work, you get the random
    assignment.

    Pros (w.r.t option 2):

      * On a new installation, all of the UIDs that are assigned will
        likely be fixed, so you get some of the benefit of the first
        option without having to create an overlay.

    Cons (w.r.t. option 2):

      * The upgrade path is a little hairier. On an existing system,
        we'll be introducing a bunch of new user packages that want
        a particular UID but don't get it. On existing systems, the
        UID assignment will still essentially be random. That's
        problematic for packages that truly need the UID they've
        requested.

Option #1 is pretty much dead in the water without an upgrade path.
We're considering #2 and #3 at the moment, but they're really not
that different.


> It could output an elog
> saying the uid/gid doesn't match the new Gentoo norm, and provide the
> commands to run to bring things into line (usermod, groupmod, find /
> -user -exec chown, etc, etc)
> 

There's no safe way to do that. If you call chown (as root) on
everything in a user-owned directory, that user can just create a
symlink to e.g. /root/.bashrc and you'll give it root. It's really a
miserable time trying to switch ownership of anything safely once it's
been created.

Re: [gentoo-dev] Requirements for UID/GID management

[A. Wilcox]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 28/01/17 13:32, James Le Cuirot wrote:
> On Sat, 28 Jan 2017 12:13:53 -0600 "A. Wilcox"
> <awilfox@adelielinux.org> wrote:
> 
>> Having a file that user.eclass would use to map new users/groups
>> to IDs would be extremely beneficial to me.  I was thinking about
>> diving in to that some time later, after the GLEP 70 work I'm
>> doing, but if someone else wants to take it - please!  That would
>> greatly ease the pain of not only NFS, but swapping data disks
>> around between different / .
>> 
>> Consider, for example, one of my use cases for this:  I have a 
>> LibreSSL / that I use solely for testing ebuilds against it, and
>> my regular / with OpenSSL.  I share /home and /srv between these
>> two, but the apache, nginx, and charybdis users have different
>> UIDs between them.  Therefore I have to chown -R each time I test
>> LibreSSL.
>> 
>> I could use a different /home and /srv, or make two copies, but
>> it's much easier for me to test these apps having my entire
>> normal environment available to me.
> 
> As mentioned in my other post, why are you not using idmapd? It's 
> trivial to set up on top of NFSv4.
> 

I think you have missed the point.  This is not on a network and this
has nothing to do with NFS of any version.

This is two LVM volumes, /dev/ciall/libressl and /dev/ciall/root,
mounting /dev/ciall/home and /dev/ciall/srv where they belong.  The
kernel is started with different root=, or sometimes I just bind mount
and chroot if I want to run both at the same time.

Nothing to do with networking, just two parallel Gentoo installs on the
same machine that can't agree worth a darn on UIDs/GIDs.

I like the pre_pkg_setup idea spouted elsewhere in this thread and will
likely start using this myself until Gentoo figures out how to guarantee
stability with UIDs.

Regards,
- --arw

- -- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJYjiOAAAoJEMspy1GSK50UWrAQAIq0fz+oxA2SWCEBNdKKZgRN
gNJOcnEj9lWSiE4bXA4C4diFj38GD5HhTK6awwNjEJ2e3S/+IvJvy97RcUjzpP09
iE+77p3YyZeVxywONJ+BK0gSWc3pJrYzzZWMmHMhIhA5TW78OPKFgP4q+FT8Ruwu
TdYL4/cH6shcELacsgLq+0fBxgT8xkl5LA0OWdW5g2lVFzcnZ7sM/qX7WMlksaVY
o4fPBc10hNwrAW5HsSBOw6tZsf+8CtaBaYVub1DfgWSLxmE70+9pX+4VCObvuc9m
CZ1u3tvcus7xBbpIDxD9M6yC7Jkj250Gr0IAJol2y8JJY5EyCu/iNtUbHT3lgb+F
qRKLbMDp91F9jzHmup04UuJdVkcDaxA4nfb7RWGOnH4U5BmmCdHkxUMtSA2vPNAh
9m7dwn+Yb6OjHKvB/gswbRfT4vV6bn0a07/J72GBgoWvEvGZ/rj9mO5O7gu/k9eQ
zXc3eSWWmi6S3FtDHKNP7U+CrBGOu6DN79GGDO4zzGpl8aWGFm8ol3qW5jtWKdsP
y0K1n2erH1Xfj5CoXzcbm3s7EQxiB3hEHlfv2gYYMrZZuqKgrVXQ5krvNApbsiQo
dVmGOugfIUnVcv/o6rpwtCzNnyGahq6PISkJbRwzh3irA7fsZjYYZITnN6Ba0jIU
EVO3/JgkJDAvyn+ZWQ8O
=aaA7
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/29/2017 05:03 AM, Ulrich Mueller wrote:
>>>>>> On Sat, 28 Jan 2017, Michael Orlitzky wrote:
> 
>> [...] sys-user/echo [...]
> 
> [Replying to a random message in this thread, as I have some backlog.]
> 
> Users and groups aren't packages, so IMHO packages and *DEPEND
> variables shouldn't be abused for things like that. This has been
> discussed in bug 53269 previously.

Well they're not packages now because they're not packages now. If we
make them packages, they'll be packages. We treat them a lot like
packages, and by making them packages, a lot of things get simpler.


> IMHO the way to go is to add proper variables in the next EAPI, like
> the EUSERS and EGROUPS proposed (and approved) in GLEP 27.

But then we need to add features to make EUSERS and EGROUPS work exactly
like DEPEND and RDEPEND, which seems counter-productive. What if I need
a user conditionally on some USE flag? What if I need to share a user
with another package? How do users get removed?

I will try to coalesce all of this into a wiki page at some point.

Re: [gentoo-dev] Requirements for UID/GID management

[A. Wilcox]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 29/01/17 11:05, Michael Orlitzky wrote:
> On 01/29/2017 03:26 AM, Alan McKinnon wrote:
>>> 
>>> Can anyone think of an upgrade path for fixed UIDs? That issue
>>> aside, I may have convinced myself that fixed UIDs are better.
>> 
>> The general process I would recommend is that if the ebuild finds
>> the user already exists, leave it, it's UID and it's file
>> ownerships alone, and keep them as they are. If the user does not
>> exist then create it.
> 
> That's what I've got it doing now...
> 
> 
>> Preferably use a pre-assigned UID/GID so there is some
>> consistency with most other Gentoo things out there.
> 
> This is the only point we have left to consider. To recap, there
> are three approaches to try:
> 
> 1 Truly fixed IDs. Every user gets the UID it wants, or it doesn't 
> get created. The UIDs are all determined beforehand.
> 
> 2 Mostly random UIDs, and the few packages that need to specify
> one can do so. Usually installation will never fail, but if some
> user specifies a particular UID and doesn't get it, we die().
> 
> 3 Mostly fixed UIDs, but with a fallback to random ones if you
> don't get the UID you want. Here, everyone specifies their
> "preferred" UID, and we try that first. If it doesn't work, you get
> the random assignment.


You could easily start with #3, and after some years, move to #1.

Anyone with a 20 year old Gentoo install (by that time) should expect
to have to do very heavy lifting.

I for one am more than willing to do whatever shell commands necessary
to make all my Gentoo installs agree on UIDs and get #1 now, but I
realise most people are not.

- --arw

- -- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJYjiTOAAoJEMspy1GSK50UCgYP/j7zBRAiL6w7fACER+A+J/3x
keXe4OsBzlNsUxqC+BrQ/Y9tCSJnIHRIs6ozQCgEdfAKJfkLqkSmKAY3O3RT+mho
VzjUCibftf/UNGOnFf6BqXCeBEjtV1YA7URlYumNyHxdG/AFIICWYFSSTLwzJoR1
91wqJmbcUI3LtQXoXodaYC2nbUWvcbO8RyxpDmxZ33L8xj1lAgpuFNcdEs+Rscxp
oDK4zJC/K8wUYTUR2YO1Lb3lPF6qgJbMcX0YpQaXIGeYA2PXf4O+LqTXmGNr4O9r
DFM3dbPgq2YPuHORACUY5YsmPBjHiaJlgzJo2WrhnIc2D1MPhA430Xlloiua3kF9
G7yqkz7mhBtJFrExoQ2MrtXMB5vwDUZ+3qrBzx/cKfxpSzsRck5NZ27eWK0oEpg2
fAUFJT7iIwSD3WyLkQbc2HHQ5nnTlnrBHM56YgCIPgz1Y4aNSB7hA+tCfQj4CNZC
Y25d9VzBM2KclASiH6ROQLK5EyU0joMtZvTRx89b8SJV+AebLeaWtCsGe41KeF/W
iDSnPGXtKRLYZtdebxGCXZwbaUVCRu/cIH2TXMpWDjm0iw3GoFZ6jiLveRCns59U
UecZNQph5tPc/HBX2zCTTmH3jNfifSfb525aHVnUSVlyTWa8SQzw2jlnOuAkI33q
8MY5++CHplEPGVCvYMrc
=99NE
-----END PGP SIGNATURE-----

Re: [gentoo-dev] Requirements for UID/GID management

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 1541 bytes --]

On Sun, 29 Jan 2017 11:16:50 -0600
"A. Wilcox" <awilfox@adelielinux.org> wrote:

> On 28/01/17 13:32, James Le Cuirot wrote:
> > On Sat, 28 Jan 2017 12:13:53 -0600 "A. Wilcox"
> > <awilfox@adelielinux.org> wrote:
> >   
> >> Having a file that user.eclass would use to map new users/groups
> >> to IDs would be extremely beneficial to me.  I was thinking about
> >> diving in to that some time later, after the GLEP 70 work I'm
> >> doing, but if someone else wants to take it - please!  That would
> >> greatly ease the pain of not only NFS, but swapping data disks
> >> around between different / .
> >> 
> >> Consider, for example, one of my use cases for this:  I have a 
> >> LibreSSL / that I use solely for testing ebuilds against it, and
> >> my regular / with OpenSSL.  I share /home and /srv between these
> >> two, but the apache, nginx, and charybdis users have different
> >> UIDs between them.  Therefore I have to chown -R each time I test
> >> LibreSSL.
> >> 
> >> I could use a different /home and /srv, or make two copies, but
> >> it's much easier for me to test these apps having my entire
> >> normal environment available to me.  
> > 
> > As mentioned in my other post, why are you not using idmapd? It's 
> > trivial to set up on top of NFSv4.
> 
> I think you have missed the point.  This is not on a network and this
> has nothing to do with NFS of any version.

Apologies, I didn't read that as closely as I should have done.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[james]

On 01/29/2017 12:22 PM, A. Wilcox wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 29/01/17 11:05, Michael Orlitzky wrote:
>> On 01/29/2017 03:26 AM, Alan McKinnon wrote:
>>>>
>>>> Can anyone think of an upgrade path for fixed UIDs? That issue
>>>> aside, I may have convinced myself that fixed UIDs are better.
>>>
>>> The general process I would recommend is that if the ebuild finds
>>> the user already exists, leave it, it's UID and it's file
>>> ownerships alone, and keep them as they are. If the user does not
>>> exist then create it.
>>
>> That's what I've got it doing now...
>>
>>
>>> Preferably use a pre-assigned UID/GID so there is some
>>> consistency with most other Gentoo things out there.
>>
>> This is the only point we have left to consider. To recap, there
>> are three approaches to try:
>>
>> 1 Truly fixed IDs. Every user gets the UID it wants, or it doesn't
>> get created. The UIDs are all determined beforehand.
>>
>> 2 Mostly random UIDs, and the few packages that need to specify
>> one can do so. Usually installation will never fail, but if some
>> user specifies a particular UID and doesn't get it, we die().
>>
>> 3 Mostly fixed UIDs, but with a fallback to random ones if you
>> don't get the UID you want. Here, everyone specifies their
>> "preferred" UID, and we try that first. If it doesn't work, you get
>> the random assignment.
>
>
> You could easily start with #3, and after some years, move to #1.

Yep.  But, why can't (1) be  selectable (now) as  part of a profile, 
once that discussion on profiles is formalized into a pathway forward?


> Anyone with a 20 year old Gentoo install (by that time) should expect
> to have to do very heavy lifting.

Just leave them alone for now, as gentoo systems can now have different 
gid/uid mappings. Migration strategies will emerge over time. We'd  need 
some mechanism to determine if a given package attempts to set a 
contrarian uid/gid.  Perhaps a flag for those packages could address 
uid/gid conflicts going forward in a one-off solution?


> I for one am more than willing to do whatever shell commands necessary
> to make all my Gentoo installs agree on UIDs and get #1 now, but I
> realise most people are not.

YES! I think after (1) is finalized, it should be part of the handbook 
installation as a default, but selectable. That way the migration
is gently fast-tracked. Matching up with Debian, is a really good idea,
as long as nothing is conflated by systemd.

> - --arw


hth,
James

>
> - --
> A. Wilcox (awilfox)
> Project Lead, Adélie Linux
> http://adelielinux.org
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBCAAGBQJYjiTOAAoJEMspy1GSK50UCgYP/j7zBRAiL6w7fACER+A+J/3x
> keXe4OsBzlNsUxqC+BrQ/Y9tCSJnIHRIs6ozQCgEdfAKJfkLqkSmKAY3O3RT+mho
> VzjUCibftf/UNGOnFf6BqXCeBEjtV1YA7URlYumNyHxdG/AFIICWYFSSTLwzJoR1
> 91wqJmbcUI3LtQXoXodaYC2nbUWvcbO8RyxpDmxZ33L8xj1lAgpuFNcdEs+Rscxp
> oDK4zJC/K8wUYTUR2YO1Lb3lPF6qgJbMcX0YpQaXIGeYA2PXf4O+LqTXmGNr4O9r
> DFM3dbPgq2YPuHORACUY5YsmPBjHiaJlgzJo2WrhnIc2D1MPhA430Xlloiua3kF9
> G7yqkz7mhBtJFrExoQ2MrtXMB5vwDUZ+3qrBzx/cKfxpSzsRck5NZ27eWK0oEpg2
> fAUFJT7iIwSD3WyLkQbc2HHQ5nnTlnrBHM56YgCIPgz1Y4aNSB7hA+tCfQj4CNZC
> Y25d9VzBM2KclASiH6ROQLK5EyU0joMtZvTRx89b8SJV+AebLeaWtCsGe41KeF/W
> iDSnPGXtKRLYZtdebxGCXZwbaUVCRu/cIH2TXMpWDjm0iw3GoFZ6jiLveRCns59U
> UecZNQph5tPc/HBX2zCTTmH3jNfifSfb525aHVnUSVlyTWa8SQzw2jlnOuAkI33q
> 8MY5++CHplEPGVCvYMrc
> =99NE
> -----END PGP SIGNATURE-----
>
>
>

Re: [gentoo-dev] Requirements for UID/GID management

[Alan McKinnon]

On 29/01/2017 19:05, Michael Orlitzky wrote:
> On 01/29/2017 03:26 AM, Alan McKinnon wrote:
>>>
>>> Can anyone think of an upgrade path for fixed UIDs? That issue aside, I
>>> may have convinced myself that fixed UIDs are better.
>>
>> The general process I would recommend is that if the ebuild finds the user
>> already exists, leave it, it's UID and it's file ownerships alone, and keep
>> them as they are. If the user does not exist then create it.
> 
> That's what I've got it doing now...
> 
> 
>> Preferably use a pre-assigned UID/GID so there is some consistency
>> with most other Gentoo things out there.
> 
> This is the only point we have left to consider. To recap, there are
> three approaches to try:


[snip]

>   3 Mostly fixed UIDs, but with a fallback to random ones if you don't
>     get the UID you want. Here, everyone specifies their "preferred"
>     UID, and we try that first. If it doesn't work, you get the random
>     assignment.
> 
>     Pros (w.r.t option 2):
> 
>       * On a new installation, all of the UIDs that are assigned will
>         likely be fixed, so you get some of the benefit of the first
>         option without having to create an overlay.

This is my preferred choice

> 
>     Cons (w.r.t. option 2):
> 
>       * The upgrade path is a little hairier. On an existing system,
>         we'll be introducing a bunch of new user packages that want
>         a particular UID but don't get it. On existing systems, the
>         UID assignment will still essentially be random. That's
>         problematic for packages that truly need the UID they've
>         requested.

I say drop the upgrade path entirely. Why do you as a dev care?
You've made a compelling case that making any changes at all is hairy,
so don't even try and automate it.

As a mere user I personally would never expect Gentoo devs to deal with
this, and if there's a knob in make.conf to enable/disable such a user
upgrade path, I'd probably disable it.

I've had to do this exercise more times than I care to admit, usually to
get multiple machines across a network in sync or to deal with nfsv3,
and it's not something I'd ever dare throw a script or ebuild at.

If you achieved the simpler status of new installs of packages get the
preferred UID if available, then I'm 100% happy and over the moon.
Everything else is my problem, not yours (although I appreciate the
effort to even make the attempt)

> Option #1 is pretty much dead in the water without an upgrade path.
> We're considering #2 and #3 at the moment, but they're really not
> that different.
> 
> 
>> It could output an elog
>> saying the uid/gid doesn't match the new Gentoo norm, and provide the
>> commands to run to bring things into line (usermod, groupmod, find /
>> -user -exec chown, etc, etc)
>>
> 
> There's no safe way to do that. If you call chown (as root) on
> everything in a user-owned directory, that user can just create a
> symlink to e.g. /root/.bashrc and you'll give it root. It's really a
> miserable time trying to switch ownership of anything safely once it's
> been created.

Sure it can be done, just don't chown -R <user> ~user. DO it the VERY
long way round, file by file. Say you changed user "awesome" uid 300 to 400:

find / -uid 300 -exec chown awesome {} \+

It works every time, but involves " find / " - ouch - and you can't do
it if the user is logged in or has running processes. This is all for
system accounts, so I reckon you have a huge show stopper right there,
requiring admin intervention

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/27/2017 12:54 PM, Michael Orlitzky wrote:
> We approved GLEP 27 (https://wiki.gentoo.org/wiki/GLEP:27) in 2004 but
> never implemented it. I'm wondering what are the explicit requirements
> that we have for user and group management?
> 
> What I'm really wondering is, instead of the proposal in GLEP27, if we
> couldn't simply handle users like any other package.

I put together a draft of the "fixed UIDs with random fallback" model:

  https://wiki.gentoo.org/wiki/User:Mjo/GLEP:User_packages

If we decide to fix UID/GID management, I think it would be a lot easier
to implement that draft than GLEP:27. I would be interested in hearing
for which use cases GLEP:27 would be preferable.

If anyone wants to play with the draft or work on a reference
implementation, I can move this stuff to a public namespace.

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/29/2017 05:07 PM, Alan McKinnon wrote:
> 
> Sure it can be done, just don't chown -R <user> ~user. DO it the VERY
> long way round, file by file. Say you changed user "awesome" uid 300 to 400:
> 
> find / -uid 300 -exec chown awesome {} \+
> 

That will find symlinks created by UID 300, and chown will follow them
to give "awesome" ownership of the TARGET of the symlink; an easy root
exploit. If you are about to suggest "find -type f" or the
"--no-dereference" flag, then beware that chown will also follow
hardlinks and you're still screwed (albeit limited to one filesystem,
and on vanilla kernels).

Re: [gentoo-dev] Requirements for UID/GID management

[Alan McKinnon]

On 30/01/2017 00:20, Michael Orlitzky wrote:
> On 01/29/2017 05:07 PM, Alan McKinnon wrote:
>>
>> Sure it can be done, just don't chown -R <user> ~user. DO it the VERY
>> long way round, file by file. Say you changed user "awesome" uid 300 to 400:
>>
>> find / -uid 300 -exec chown awesome {} \+
>>
> 
> That will find symlinks created by UID 300, and chown will follow them
> to give "awesome" ownership of the TARGET of the symlink; an easy root
> exploit. If you are about to suggest "find -type f" or the
> "--no-dereference" flag, then beware that chown will also follow
> hardlinks and you're still screwed (albeit limited to one filesystem,
> and on vanilla kernels).
> 
> 


Good catch with symlinks.
I don't see the point about hardlinks, they are just files with 2
dentries. When find gets to the second one it's already changed, so no
problem.

But I'm sure there are plenty edge case scenarios that make this whole
process go awry, all pointing to the same conclusion:

As a dev you shouldn't even try. Let the sysadmin deal with it.
If a system user already has a UID different to the published standard,
leave it alone, it's a human's problem

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/29/2017 05:30 PM, Alan McKinnon wrote:
> 
> Good catch with symlinks.
> I don't see the point about hardlinks, they are just files with 2
> dentries. When find gets to the second one it's already changed, so no
> problem.
> 

Any user can create a hard link in its home directory to /etc/shadow, so
long as (a) they live on the same filesystem, and (b) there are no
special kernel protections in place to prevent it. If you call chown on
that hard link, it will change the ownership of /etc/shadow.

I thought real hard about ways to avoid that and ultimately gave up. The
only safe way to chown is to "chown away"; that is, switch to the guy
who owns the files, and then give them to someone else.

Re: [gentoo-dev] Requirements for UID/GID management

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1031 bytes --]

>>>>> On Sun, 29 Jan 2017, Michael Orlitzky wrote:

> I put together a draft of the "fixed UIDs with random fallback" model:

>   https://wiki.gentoo.org/wiki/User:Mjo/GLEP:User_packages

> If we decide to fix UID/GID management, I think it would be a lot
> easier to implement that draft than GLEP:27. I would be interested
> in hearing for which use cases GLEP:27 would be preferable.

Our syntax for package names is more restrictive than what POSIX
allows for a portable user name. Therefore, there could be user names
that are not representable. Have you checked if all user and group
names currently in use (at least in the main tree) also follow the
rules for package names? IIRC, at some point in time we had names
starting with "_" (which is ugly, but would be allowed as a package
name) and some "foo-1" type names (which are not allowed).

What about duplicates turning up in searches for packages? For example,
"emerge portage" won't work any more because there would be the ones
in sys-group and sys-user too.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/29/2017 06:34 PM, Ulrich Mueller wrote:
> 
> Our syntax for package names is more restrictive than what POSIX
> allows for a portable user name. Therefore, there could be user names
> that are not representable. Have you checked if all user and group
> names currently in use (at least in the main tree) also follow the
> rules for package names? IIRC, at some point in time we had names
> starting with "_" (which is ugly, but would be allowed as a package
> name) and some "foo-1" type names (which are not allowed).

There's already a variable SYS_USER_NAME=${PN}. If it turns out there
are unrepresentable names, we can change that to a default value and let
people change it in the ebuild. The main benefit of having the username
be the package name is that it prevents duplicates without any
additional code or convention.


> What about duplicates turning up in searches for packages? For example,
> "emerge portage" won't work any more because there would be the ones
> in sys-group and sys-user too.

Isn't that why we have categories in the first place?

Re: [gentoo-dev] Requirements for UID/GID management

[Alan McKinnon]

On 30/01/2017 01:04, Michael Orlitzky wrote:
> On 01/29/2017 05:30 PM, Alan McKinnon wrote:
>>
>> Good catch with symlinks.
>> I don't see the point about hardlinks, they are just files with 2
>> dentries. When find gets to the second one it's already changed, so no
>> problem.
>>
> 
> Any user can create a hard link in its home directory to /etc/shadow, so
> long as (a) they live on the same filesystem, and (b) there are no
> special kernel protections in place to prevent it. If you call chown on
> that hard link, it will change the ownership of /etc/shadow.

That is absolutely not true, at least for the case of classic Unix
filesystems.

hardlinks are exactly the same thing as regular files. For any given
filesystem object there is a dentry, and that dentry points to an inode.
Usually that is the end of the matter.

When we create hardlinked files all we are doing is create a new dentry
and point it to an inode that is already there. The so-called
"hardlinked" file is a fiction, the instant you do it the new dentry
operates just like any other file and is not even aware of other
dentries pointing to the same inode.

The point being, there is only one inode, and that is where the
ownerships and permissions are. I cannot chmod, chown or chgrp
/etc/shadow because I do not own it, and the kernel will not let me ln
it either:

alan@khamul /alan $ ls -ald /alan/
drwxr-xr-x 2 alan root 4096 Jan 30 16:10 /alan/
alan@khamul /alan $ ln /etc/shadow
ln: failed to create hard link './shadow' => '/etc/shadow': Operation
not permitted
alan@khamul /alan $ ls -al /etc/shadow
-rw-r----- 1 root root 1655 Dec 31 14:43 /etc/shadow
alan@khamul /alan $ stat /etc/shadow
  File: /etc/shadow
  Size: 1655            Blocks: 8          IO Block: 4096   regular file
Device: 815h/2069d      Inode: 1188230     Links: 1
Access: (0640/-rw-r-----)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-12-31 14:43:29.556174143 +0200
Modify: 2016-12-31 14:43:29.556174143 +0200
Change: 2016-12-31 14:43:29.568174144 +0200
 Birth: -

The only thing I can do after hardlinking a file is what I could do before.

> I thought real hard about ways to avoid that and ultimately gave up. The
> only safe way to chown is to "chown away"; that is, switch to the guy
> who owns the files, and then give them to someone else.

This is also not true.

Only root can chown the owner of a file, and a regular user cannot give
files
away. The only ownership actions a user can do on a file is chgrp but
only if
the user is the owner, and then only to a group the user is a member of.



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/30/2017 09:25 AM, Alan McKinnon wrote:
>>
>> Any user can create a hard link in its home directory to /etc/shadow, so
>> long as (a) they live on the same filesystem, and (b) there are no
>> special kernel protections in place to prevent it. If you call chown on
>> that hard link, it will change the ownership of /etc/shadow.
> 
> That is absolutely not true, at least for the case of classic Unix
> filesystems.
> 
> ...
> 
> I cannot chmod, chown or chgrp
> /etc/shadow because I do not own it, and the kernel will not let me ln
> it either:
> 
> alan@khamul /alan $ ln /etc/shadow
> ln: failed to create hard link './shadow' => '/etc/shadow': Operation
> not permitted
> 

You have the fs.protected_hardlinks sysctl enabled. We patch that in
gentoo-sources, but it's off by default in vanilla-sources. Try again
with it disabled (and don't forget to turn it back on). Once the hard
link has been created, a "chown -R foo /alan" or the equivalent "find
..." command will change the ownership of /etc/shadow.

Re: [gentoo-dev] Requirements for UID/GID management

[Patrick McLean]

On Mon, 30 Jan 2017 11:29:02 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 01/30/2017 09:25 AM, Alan McKinnon wrote:
> >>
> >> Any user can create a hard link in its home directory
> >> to /etc/shadow, so long as (a) they live on the same filesystem,
> >> and (b) there are no special kernel protections in place to
> >> prevent it. If you call chown on that hard link, it will change
> >> the ownership of /etc/shadow.  
> > 
> > That is absolutely not true, at least for the case of classic Unix
> > filesystems.
> > 
> > ...
> > 
> > I cannot chmod, chown or chgrp
> > /etc/shadow because I do not own it, and the kernel will not let me
> > ln it either:
> > 
> > alan@khamul /alan $ ln /etc/shadow
> > ln: failed to create hard link './shadow' => '/etc/shadow':
> > Operation not permitted
> >   
> 
> You have the fs.protected_hardlinks sysctl enabled. We patch that in
> gentoo-sources, but it's off by default in vanilla-sources. Try again
> with it disabled (and don't forget to turn it back on). Once the hard
> link has been created, a "chown -R foo /alan" or the equivalent "find
> ..." command will change the ownership of /etc/shadow.
> 
> 

No, that is also enabled by default on vanilla kernels, I just verified
on my machine running a vanilla kernel. It doesn't matter anyway, since
the permissions and ownership information is stored in the inode, not
the dentry so all hardlinks have exactly the same permissions.

Re: [gentoo-dev] Requirements for UID/GID management

[Michael Orlitzky]

On 01/30/2017 01:05 PM, Patrick McLean wrote:
> 
> No, that is also enabled by default on vanilla kernels, I just verified
> on my machine running a vanilla kernel. It doesn't matter anyway, since
> the permissions and ownership information is stored in the inode, not
> the dentry so all hardlinks have exactly the same permissions.
> 

I don't believe you =P

Check https://github.com/torvalds/linux/blob/master/fs/namei.c:

  int sysctl_protected_symlinks __read_mostly = 0;
  int sysctl_protected_hardlinks __read_mostly = 0;

And compare with:

https://gitweb.gentoo.org/proj/linux-patches.git/tree/1510_fs-enable-link-security-restrictions-by-default.patch?h=4.9

The fact that all permission and ownership information is shared is
precisely the problem. When you change ownership of the hardlink (which
you'll never know is a hardlink), you change ownership of /etc/shadow.

Re: [gentoo-dev] Requirements for UID/GID management

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 2129 bytes --]

On 01/30/2017 07:22 PM, Michael Orlitzky wrote:
> On 01/30/2017 01:05 PM, Patrick McLean wrote:
>>
>> No, that is also enabled by default on vanilla kernels, I just verified
>> on my machine running a vanilla kernel. It doesn't matter anyway, since
>> the permissions and ownership information is stored in the inode, not
>> the dentry so all hardlinks have exactly the same permissions.
>>
> 
> I don't believe you =P
> 
> Check https://github.com/torvalds/linux/blob/master/fs/namei.c:
> 
>   int sysctl_protected_symlinks __read_mostly = 0;
>   int sysctl_protected_hardlinks __read_mostly = 0;
> 
> And compare with:
> 
> https://gitweb.gentoo.org/proj/linux-patches.git/tree/1510_fs-enable-link-security-restrictions-by-default.patch?h=4.9
> 
> The fact that all permission and ownership information is shared is
> precisely the problem. When you change ownership of the hardlink (which
> you'll never know is a hardlink), you change ownership of /etc/shadow.
> 
> 

To provide some background for this, it was included in mainstream
kernel at one point but caused userland regression in some edge cases so
was removed again.

It is already discussed at least on [0] and it seems the behavior was
turned the other way around in [1]: "In commit 800179c9b8a1 ("This adds
symlink and hardlink restrictions to
the Linux VFS"), the new link protections were enabled by default, in
the hope that no actual application would care, despite it being
technically against legacy UNIX (and documented POSIX) behavior.

However, it does turn out to break some applications.  It's rare, and
it's unfortunate, but it's unacceptable to break existing systems, so
we'll have to default to legacy behavior.
"

You'll find some more discussion around this in e.g [bug 540006]

References:
[0] http://lwn.net/Articles/521626/
[1] http://www.spinics.net/lists/stable-commits/msg21052.html
[bug 540006] https://bugs.gentoo.org/show_bug.cgi?id=540006

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[gentoo-dev] Re: Requirements for UID/GID management

[Martin Vaeth]

Michael Orlitzky <mjo@gentoo.org> wrote:
>
> The fact that all permission and ownership information is shared is
> precisely the problem. When you change ownership of the hardlink (which
> you'll never know is a hardlink), you change ownership of /etc/shadow.

Why should this be a problem except for a race between reading
and changing the ownership?
Admittedly, by using "find ... -exec ... +" the time for an exploit
of the race is even increased when a "standard" chown command is used.

However, it is no rocket science to write a race-free chown command
in C: Just open the file and use stat() and fchown() to be sure to
change only files from the "correct" user.

Since this works on the filehandle and not on the filename, I think
that there is no possibility for an exploit when this is used in the
above find loop.

Re: [gentoo-dev] Re: Requirements for UID/GID management

[Michael Orlitzky]

On 02/03/2017 09:51 AM, Martin Vaeth wrote:
> Michael Orlitzky <mjo@gentoo.org> wrote:
>>
>> The fact that all permission and ownership information is shared is
>> precisely the problem. When you change ownership of the hardlink (which
>> you'll never know is a hardlink), you change ownership of /etc/shadow.
> 
> Why should this be a problem except for a race between reading
> and changing the ownership?
> Admittedly, by using "find ... -exec ... +" the time for an exploit
> of the race is even increased when a "standard" chown command is used.
> 
> However, it is no rocket science to write a race-free chown command
> in C: Just open the file and use stat() and fchown() to be sure to
> change only files from the "correct" user.
> 
> Since this works on the filehandle and not on the filename, I think
> that there is no possibility for an exploit when this is used in the
> above find loop.

Not a bad idea... we chould ship that safe-chown utility, and then tell
users how to use it to fix their UIDs. The draft that I wrote up was for
the "fixed UID with random fallback" model, but said utility could still
be useful for people who want to change their running systems to use the
same UIDs that would have been chosen by default.

Re: [gentoo-dev] Re: Requirements for UID/GID management

[Christopher Head]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

On Fri, 3 Feb 2017 14:29:04 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:

> > However, it is no rocket science to write a race-free chown command
> > in C: Just open the file and use stat() and fchown() to be sure to
> > change only files from the "correct" user.
> > 
> > Since this works on the filehandle and not on the filename, I think
> > that there is no possibility for an exploit when this is used in the
> > above find loop.  
> 
> Not a bad idea... we chould ship that safe-chown utility, and then
> tell users how to use it to fix their UIDs. The draft that I wrote up
> was for the "fixed UID with random fallback" model, but said utility
> could still be useful for people who want to change their running
> systems to use the same UIDs that would have been chosen by default.

Are you sure that said utility isn’t simply “chown --from”?
-- 
Christopher Head

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 630 bytes --]

Re: [gentoo-dev] Re: Requirements for UID/GID management

[Michael Orlitzky]

On 02/04/2017 03:50 AM, Christopher Head wrote:
>>
>> Not a bad idea... we chould ship that safe-chown utility, and then
>> tell users how to use it to fix their UIDs. The draft that I wrote up
>> was for the "fixed UID with random fallback" model, but said utility
>> could still be useful for people who want to change their running
>> systems to use the same UIDs that would have been chosen by default.
> 
> Are you sure that said utility isn’t simply “chown --from”?
> 

One would hope that if they implemented it, that they did it safely --
but alas it's not standard.

[gentoo-dev] Re: Requirements for UID/GID management

[Martin Vaeth]

Christopher Head <chead@chead.ca> wrote:
>
> Are you sure that said utility isn't simply chown --from=?

As usual, I just checked the POSIX standard and not the
GNU extensions before posting  ;)

I did now a quick audit of the coreutils-8.25 source:
It seems to be safely implemented in the way I mentioned.