* [gentoo-dev] Modification proposal for user/group creation when ROOT!="/"
[not found] <1187243158.3442649.1464116425074.JavaMail.yahoo.ref@mail.yahoo.com>
@ 2016-05-24 19:00 ` Farid BENAMROUCHE
2016-06-08 5:44 ` [gentoo-dev] " Benda Xu
0 siblings, 1 reply; 8+ messages in thread
From: Farid BENAMROUCHE @ 2016-05-24 19:00 UTC (permalink / raw
To: gentoo-dev
Hi all,
Currently there is an old known limitation when using ROOT= option to install a package in a folder: user/groups are created in the host filesystem, not the target root filesystem.
So I've pushed some modifications to the upstream shadow repo.
Basically, I've added a --prefix option to user{add,mod,del} and group{add,mod,del}
This option does the same as --root option, but whithout a chroot (so compatible when cross compiling)
You can see more details (and the limitation of my implementation) in the shadow github repo:
https://github.com/shadow-maint/shadow/issues/18
Now, for the gentoo part, I do have a working solution that I've pushed in the following bugzilla:
https://bugs.gentoo.org/show_bug.cgi?id=541406
A new user.eclass file with modified enewuser,enewgroup and egetent that all supports ${ROOT} option via --prefix in shadow utilities.
For now I've only added this option for linux.
However, I've encountered some unexpected issues: some ebuilds are using direct calls to chown and fowners. Both are not compatible with ${ROOT}...
To solve this, I've created 2 new calls in user.eclass: echown and efowners.
The only thing the new functions are doing is to get the uid/gid from the correct passwd/group files from ${ROOT} using the modified egetent function and pass that to the native chown/fowners...
For example, in sys-power/nut we can find:
chown nut:nut ${ROOT}/var/lib/nut
This should be changed to
echown nut:nut ${ROOT}/var/lib/nut
Same to fowners.
If the modification is not done, either the ebuild will fail because the nut user does not exists in the host, or the incorrect uid will be user in ${ROOT}
The solution is not perfect, but at least better than what we have today, and totally usable I believe.
I've uploaded the patches for lighttpd and nut, plus my patch for user.eclass for review in this bug... we do have time until upstream shadow team reviews and commits my modifications (at least).
Side note: it's a bit complicated to know when to add ${ROOT} and when not in a ebuild... For example, chown needs ${ROOT} but fowners must not!...
Side note 2: maybe I should add a verification to check if useradd/groupadd supports my new --prefix solution, and fallback to original behavior if not?
Tests: I've compiled a full working system with my above solution, and it works. (cross compilation in a dedicated target root path)
Regards,
Farid
^ permalink raw reply [flat|nested] 8+ messages in thread
* [gentoo-dev] Re: Modification proposal for user/group creation when ROOT!="/"
2016-05-24 19:00 ` [gentoo-dev] Modification proposal for user/group creation when ROOT!="/" Farid BENAMROUCHE
@ 2016-06-08 5:44 ` Benda Xu
0 siblings, 0 replies; 8+ messages in thread
From: Benda Xu @ 2016-06-08 5:44 UTC (permalink / raw
To: gentoo-dev
Hi Farid,
This is an excellent idea! It is very helpful for Gentoo Prefix/libc,
where we maintain a set of nss databases (passwd, group, shadow, etc.)
inside a directory prefix.
Farid BENAMROUCHE <fariouche@yahoo.fr> writes:
> Currently there is an old known limitation when using ROOT= option to
> install a package in a folder: user/groups are created in the host
> filesystem, not the target root filesystem.
Exactly.
> So I've pushed some modifications to the upstream shadow repo.
> Basically, I've added a --prefix option to user{add,mod,del} and
> group{add,mod,del} This option does the same as --root option, but
> whithout a chroot (so compatible when cross compiling)
Cool.
> You can see more details (and the limitation of my implementation) in
> the shadow github repo:
> https://github.com/shadow-maint/shadow/issues/18
Hope the upstream accept your patch soon.
> Now, for the gentoo part, I do have a working solution that I've
> pushed in the following bugzilla:
> https://bugs.gentoo.org/show_bug.cgi?id=541406
>
> A new user.eclass file with modified enewuser,enewgroup and egetent
> that all supports ${ROOT} option via --prefix in shadow utilities.
> For now I've only added this option for linux.
The new user.eclass requires the new shadow. After the upstream makes a
new release, it will take a long time for Gentoo to use the feature.
Because we should carefully handle the compatibility with the old
systems.
> However, I've encountered some unexpected issues: some ebuilds are
> using direct calls to chown and fowners. Both are not compatible with
> ${ROOT}...
Those ebuilds are broken and should be fixed.
> To solve this, I've created 2 new calls in user.eclass: echown and
> efowners. The only thing the new functions are doing is to get the
> uid/gid from the correct passwd/group files from ${ROOT} using the
> modified egetent function and pass that to the native chown/fowners...
>
> For example, in sys-power/nut we can find: chown nut:nut
> ${ROOT}/var/lib/nut
>
> This should be changed to echown nut:nut ${ROOT}/var/lib/nut
Brilliant.
> Same to fowners. If the modification is not done, either the ebuild
> will fail because the nut user does not exists in the host, or the
> incorrect uid will be user in ${ROOT}
>
> The solution is not perfect, but at least better than what we have
> today, and totally usable I believe.
>
> I've uploaded the patches for lighttpd and nut, plus my patch for
> user.eclass for review in this bug... we do have time until upstream
> shadow team reviews and commits my modifications (at least).
>
> Side note: it's a bit complicated to know when to add ${ROOT} and when
> not in a ebuild... For example, chown needs ${ROOT} but fowners must
> not!...
Why not? Could you give more details?
> Side note 2: maybe I should add a verification to check if
> useradd/groupadd supports my new --prefix solution, and fallback to
> original behavior if not?
IMHO, useradd/groupadd supporting --prefix will be captured by a new
EAPI in the (far) future. At present we don't need to worry about it.
> Tests: I've compiled a full working system with my above solution, and
> it works. (cross compilation in a dedicated target root path)
Do you have an overlay for me reproduce your result? I am also
interested in hosting it in proj/android.git[1] if you do not have one
yet.
Keep on your good work.
Yours,
Benda
1. https://gitweb.gentoo.org/proj/android.git/
^ permalink raw reply [flat|nested] 8+ messages in thread
* [gentoo-dev] Re : Modification proposal for user/group creation when ROOT!="/"
[not found] <160839029.2375229.1522435938193.ref@mail.yahoo.com>
@ 2018-03-30 18:52 ` Farid BENAMROUCHE
2018-03-30 19:23 ` James Le Cuirot
0 siblings, 1 reply; 8+ messages in thread
From: Farid BENAMROUCHE @ 2018-03-30 18:52 UTC (permalink / raw
To: gentoo-dev, Farid BENAMROUCHE
Hi all,
Already two years....
Yes, two years ago I've posted here to notify you about the creattion of users and groups when using "ROOT=".
As a reminder, if you currently emerge a package to a specific rootfs folder, some packages will actually not create the user and groups correctly inside this rootfs.
It will also not check for the existance of the user/group inside of the rootfs.
Everytime, it will check "/".
This very old gentoo issue (I have to find again the GLEP talking about this issue).
The solution is not possible without changing the behaviour of the tools used by portage. For example, portage is using shadow in most systems (and shadow is using the glibc).
I've submited a patch first to shadow upstream team to avec this feature, using a new argument (--prefix) to tools like usemod,useradd etc
You can create users in any rootfs you want, no need to be root (except if you local pam configuration requires it)
This is unfortunately not enough for gentoo: some packages are relying on chaonw directly. So I've added some wrappers (same as the current enewuser etc functions)
This enable people to build embedded systems rootfs from scratch, using a complete gentoo.
Some packages must be updated in order to behave properly in that case (replace chown with echown for example)
As a failsafe, I'm only triggering the new behavior in case ROOT env var is not "/" (and I think (I have to check) that if the command fails, it will fallback to the old behavior in case the --prefix argument is not implemented (incorrect shadow version for example)
This requires in that case a new EAPI I think. Just tell me what to do, where, and I will submit the proposal
I'm using the modifications for 2 years now.
Limitations:
- selinux is not supported (ie: you will still use the system's selinux.).
- Also, this only works with gentoo systems using the shadow package (the exact version will depend on when the new release will be done).
- Last one: I only tested on a linux configuration
Cheers
--------------------------------------------
En date de : Mar 24.5.16, Farid BENAMROUCHE <fariouche@yahoo.fr> a écrit :
Objet: Modification proposal for user/group creation when ROOT!="/"
À: gentoo-dev@lists.gentoo.org
Date: Mardi 24 mai 2016, 21h00
Hi all,
Currently there is an old known
limitation when using ROOT= option to install a package in a
folder: user/groups are created in the host filesystem, not
the target root filesystem.
So I've pushed some modifications to
the upstream shadow repo.
Basically, I've added a --prefix option
to user{add,mod,del} and group{add,mod,del}
This option does the same as --root
option, but whithout a chroot (so compatible when cross
compiling)
You can see more details (and the
limitation of my implementation) in the shadow github repo:
https://github.com/shadow-maint/shadow/issues/18
Now, for the gentoo part, I do have a
working solution that I've pushed in the following
bugzilla:
https://bugs.gentoo.org/show_bug.cgi?id=541406
A new user.eclass file with modified
enewuser,enewgroup and egetent that all supports ${ROOT}
option via --prefix in shadow utilities.
For now I've only added this option for
linux.
However, I've encountered some
unexpected issues: some ebuilds are using direct calls to
chown and fowners. Both are not compatible with ${ROOT}...
To solve this, I've created 2 new calls
in user.eclass: echown and efowners.
The only thing the new functions are
doing is to get the uid/gid from the correct passwd/group
files from ${ROOT} using the modified egetent function and
pass that to the native chown/fowners...
For example, in sys-power/nut we can
find:
chown nut:nut ${ROOT}/var/lib/nut
This should be changed to
echown nut:nut ${ROOT}/var/lib/nut
Same to fowners.
If the modification is not done, either
the ebuild will fail because the nut user does not exists in
the host, or the incorrect uid will be user in ${ROOT}
The solution is not perfect, but at
least better than what we have today, and totally usable I
believe.
I've uploaded the patches for lighttpd
and nut, plus my patch for user.eclass for review in this
bug... we do have time until upstream shadow team reviews
and commits my modifications (at least).
Side note: it's a bit complicated to
know when to add ${ROOT} and when not in a ebuild... For
example, chown needs ${ROOT} but fowners must not!...
Side note 2: maybe I should add a
verification to check if useradd/groupadd supports my new
--prefix solution, and fallback to original behavior if
not?
Tests: I've compiled a full working
system with my above solution, and it works. (cross
compilation in a dedicated target root path)
Regards,
Farid
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-dev] Re : Modification proposal for user/group creation when ROOT!="/"
2018-03-30 18:52 ` [gentoo-dev] Re : " Farid BENAMROUCHE
@ 2018-03-30 19:23 ` James Le Cuirot
2018-03-30 19:47 ` James Le Cuirot
0 siblings, 1 reply; 8+ messages in thread
From: James Le Cuirot @ 2018-03-30 19:23 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 2427 bytes --]
On Fri, 30 Mar 2018 18:52:18 +0000 (UTC)
Farid BENAMROUCHE <fariouche@yahoo.fr> wrote:
> Yes, two years ago I've posted here to notify you about the creattion of users and groups when using "ROOT=".
> As a reminder, if you currently emerge a package to a specific rootfs folder, some packages will actually not create the user and groups correctly inside this rootfs.
> It will also not check for the existance of the user/group inside of the rootfs.
> Everytime, it will check "/".
>
> This very old gentoo issue (I have to find again the GLEP talking about this issue).
>
> The solution is not possible without changing the behaviour of the tools used by portage. For example, portage is using shadow in most systems (and shadow is using the glibc).
Hi,
I have an interest this and was one of the early commenters in bug
#541406. I made my own suggestions about how this might work. Your
solution is cleaner in that it doesn't require modifying the users in
the / system but it does require significant changes to tools,
eclasses, and ebuilds so I'm on the fence about it.
I did just have a lightbulb moment though. I've been playing with
unshare recently and I wondered if we could leverage it here. First I
tried this.
$ sudo unshare -m /bin/sh -c "mount --bind /mnt/somewhere/etc/group /etc/group && groupadd foo"
groupadd: failure while writing changes to /etc/group
It is possible to bind mount individual files but it doesn't work here
because it tries to rename /etc/group to made a backup. Next I tried
the whole directory but it gives a strange error.
$ sudo unshare -m /bin/sh -c "mount --bind /mnt/somewhere/etc /etc && groupadd foo"
groupadd: Cannot determine your user name.
This reveals more.
$ sudo unshare -m /bin/sh -c "id && mount --bind /mnt/utilite/mnt/moi/etc /etc && id"
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
uid=0 gid=0 groups=0,1,2,3,4,6,10,11,20,26,27
I'm not sure why the IDs break like this and strace doesn't make it any
clearer. This seems like a route worth pursuing though because you
could create a bunch of wrappers for useradd, groupadd, chown and so on
and it would then all work transparently, even when not using the
eclass functions.
Regards,
--
James Le Cuirot (chewi)
Gentoo Linux Developer
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-dev] Re : Modification proposal for user/group creation when ROOT!="/"
2018-03-30 19:23 ` James Le Cuirot
@ 2018-03-30 19:47 ` James Le Cuirot
2018-03-30 19:56 ` James Le Cuirot
0 siblings, 1 reply; 8+ messages in thread
From: James Le Cuirot @ 2018-03-30 19:47 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 618 bytes --]
On Fri, 30 Mar 2018 20:23:49 +0100
James Le Cuirot <chewi@gentoo.org> wrote:
> I did just have a lightbulb moment though. I've been playing with
> unshare recently and I wondered if we could leverage it here.
>
> $ sudo unshare -m /bin/sh -c "mount --bind /mnt/somewhere/etc /etc && groupadd foo"
> groupadd: Cannot determine your user name.
Aha! I was trying to do this against an NFS share for a system with a
different architecture. If I use a local mount with a compatible
architecture, it actually does work. I'll explore this some more.
--
James Le Cuirot (chewi)
Gentoo Linux Developer
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-dev] Re : Modification proposal for user/group creation when ROOT!="/"
2018-03-30 19:47 ` James Le Cuirot
@ 2018-03-30 19:56 ` James Le Cuirot
0 siblings, 0 replies; 8+ messages in thread
From: James Le Cuirot @ 2018-03-30 19:56 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 955 bytes --]
On Fri, 30 Mar 2018 20:47:20 +0100
James Le Cuirot <chewi@gentoo.org> wrote:
> On Fri, 30 Mar 2018 20:23:49 +0100
> James Le Cuirot <chewi@gentoo.org> wrote:
>
> > I did just have a lightbulb moment though. I've been playing with
> > unshare recently and I wondered if we could leverage it here.
> >
> > $ sudo unshare -m /bin/sh -c "mount --bind /mnt/somewhere/etc /etc && groupadd foo"
> > groupadd: Cannot determine your user name.
>
> Aha! I was trying to do this against an NFS share for a system with a
> different architecture. If I use a local mount with a compatible
> architecture, it actually does work. I'll explore this some more.
Figured it out! The system I was doing this against has an ancient
glibc (long story) with an old nsswitch.conf. I replaced this file with
a newer one and it all started working. Do you agree this could be the
way forwards?
--
James Le Cuirot (chewi)
Gentoo Linux Developer
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-dev] Re : Modification proposal for user/group creation when ROOT!="/"
[not found] <211710464.79357.1522489187353.ref@mail.yahoo.com>
@ 2018-03-31 9:39 ` Farid BENAMROUCHE
2018-03-31 10:06 ` James Le Cuirot
0 siblings, 1 reply; 8+ messages in thread
From: Farid BENAMROUCHE @ 2018-03-31 9:39 UTC (permalink / raw
To: gentoo-dev
interresting aproach.
this could work. however, i can see a few limitations:
- you must be root.
- this is specific to linux as of today.
- if you want to hide the mechanism, i don't see how without doing the same portage modifications as in my solution.
but this is maybe worth investigating. my solution isn't perfect too, I admit.
--------------------------------------------
En date de : Ven 30.3.18, James Le Cuirot <chewi@gentoo.org> a écrit :
Objet: Re: [gentoo-dev] Re : Modification proposal for user/group creation when ROOT!="/"
À: gentoo-dev@lists.gentoo.org
Date: Vendredi 30 mars 2018, 21h56
On Fri, 30 Mar 2018 20:47:20
+0100
James Le Cuirot <chewi@gentoo.org>
wrote:
> On Fri, 30 Mar
2018 20:23:49 +0100
> James Le Cuirot
<chewi@gentoo.org>
wrote:
>
> > I did
just have a lightbulb moment though. I've been playing
with
> > unshare recently and I
wondered if we could leverage it here.
>
>
> > $ sudo unshare -m /bin/sh
-c "mount --bind /mnt/somewhere/etc /etc &&
groupadd foo"
> > groupadd:
Cannot determine your user name.
>
> Aha! I was trying to do this against an
NFS share for a system with a
> different
architecture. If I use a local mount with a compatible
> architecture, it actually does work.
I'll explore this some more.
Figured it out! The system I was doing this
against has an ancient
glibc (long story)
with an old nsswitch.conf. I replaced this file with
a newer one and it all started working. Do you
agree this could be the
way forwards?
--
James Le Cuirot (chewi)
Gentoo Linux
Developer
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [gentoo-dev] Re : Modification proposal for user/group creation when ROOT!="/"
2018-03-31 9:39 ` Farid BENAMROUCHE
@ 2018-03-31 10:06 ` James Le Cuirot
0 siblings, 0 replies; 8+ messages in thread
From: James Le Cuirot @ 2018-03-31 10:06 UTC (permalink / raw
To: gentoo-dev
[-- Attachment #1: Type: text/plain, Size: 1285 bytes --]
On Sat, 31 Mar 2018 09:39:47 +0000 (UTC)
Farid BENAMROUCHE <fariouche@yahoo.fr> wrote:
> interresting aproach.
> this could work. however, i can see a few limitations:
> - you must be root.
Actually you don't if you add -r to unshare, which gives you what is
sometimes called fakeroot. Obviously you still can't modify the files
if they are really owned by root but that's true of any solution.
> - this is specific to linux as of today.
True and I am only interested in Linux but I like to play nice. Other
platforms could potentially still briefly bind mount but it wouldn't be
isolated from the other processes so it wouldn't be entirely safe. Safe
enough though? You'd need to weigh this up against how many people use
ROOT!=/ on other platforms. Not many at all, I imagine.
> - if you want to hide the mechanism, i don't see how without doing
> the same portage modifications as in my solution.
You could handle this in the eclass functions but as you pointed out,
many things call chown/chgrp directly. Usage by ebuilds themselves can
be addressed but if a build system calls these then eclass functions
will not help. What would work is adding some identically-named
wrappers to the PATH.
--
James Le Cuirot (chewi)
Gentoo Linux Developer
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 981 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-03-31 10:06 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1187243158.3442649.1464116425074.JavaMail.yahoo.ref@mail.yahoo.com>
2016-05-24 19:00 ` [gentoo-dev] Modification proposal for user/group creation when ROOT!="/" Farid BENAMROUCHE
2016-06-08 5:44 ` [gentoo-dev] " Benda Xu
[not found] <160839029.2375229.1522435938193.ref@mail.yahoo.com>
2018-03-30 18:52 ` [gentoo-dev] Re : " Farid BENAMROUCHE
2018-03-30 19:23 ` James Le Cuirot
2018-03-30 19:47 ` James Le Cuirot
2018-03-30 19:56 ` James Le Cuirot
[not found] <211710464.79357.1522489187353.ref@mail.yahoo.com>
2018-03-31 9:39 ` Farid BENAMROUCHE
2018-03-31 10:06 ` James Le Cuirot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox