[gentoo-dev] RFC: Userkit.eclass

[gentoo-dev] RFC: Userkit.eclass

[Manuel Rüger]

[-- Attachment #1.1: Type: text/plain, Size: 1943 bytes --]

Hi everyone,

I have not started to write it, but I am considering it and rather want
to gather feedback on my idea first.
I am aware that https://wiki.gentoo.org/wiki/GLEP:27 exists, but as of
right now I haven't seen anyone working on it. The goal of this eclass
is to improve user/group handling without touching the PMS.

tl;dr: Userkit eclass will improve the user handling by externalizing
the configuration to variables that can be set from outside of the ebuild.

Userkit.eclass will inherit user.eclass and require bash arrays
USERKIT_USER and USERKIT_GROUP for configuration.
I will export a pkg_setup with the corresponding setup (basically
calling enewuser and enewgroup). It will provide get_user, get_uid,
get_group, get_gid and get_home functions.
This would allow to do something like "fowners $(get_user):$(get_group)
foo".

If ${CATEGORY}-${PN}_user and ${CATEGORY}-${PN}_group are set, these
will replace the contents of USERKIT_USER and USERKIT_GROUP, allowing
the user to fully define everything user/group related.

It will also be possible to enable a switch to that makes the ebuild
fail if those are not set, as you then can set those variables first.
Another one allows to make them nops (which is nice for testing the
ebuild via "ebuild $PN test").

My only concerns right now are:
Where to store those ${CATEGORY}-${PN}_user and ${CATEGORY}-${PN}_group?
One solution could be to have another eclass named userkit-data.eclass,
which is empty by default and needs to be forked to an overlay and then
use the eclass-override setting in repos.conf. Unfortunately this will
cause a lot of md5-cache rewrites.
Another solution would be portage/package.env or portage/env.

What happens if the ebuild wants to create multiple users/group?
Currently, I want to ignore that case and focus on the 80% ebuilds that
can profit from such an eclass.

Any thoughts?

Cheers,

Manuel


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 603 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2074 bytes --]

On Wed, 23 Nov 2016 09:44:33 +0100
Manuel Rüger <mrueg@gentoo.org> wrote:

> I have not started to write it, but I am considering it and rather want
> to gather feedback on my idea first.
> I am aware that https://wiki.gentoo.org/wiki/GLEP:27 exists, but as of
> right now I haven't seen anyone working on it. The goal of this eclass
> is to improve user/group handling without touching the PMS.
> 
> tl;dr: Userkit eclass will improve the user handling by externalizing
> the configuration to variables that can be set from outside of the ebuild.
> 
> Userkit.eclass will inherit user.eclass and require bash arrays
> USERKIT_USER and USERKIT_GROUP for configuration.
> I will export a pkg_setup with the corresponding setup (basically
> calling enewuser and enewgroup). It will provide get_user, get_uid,
> get_group, get_gid and get_home functions.
> This would allow to do something like "fowners $(get_user):$(get_group)
> foo".
> 
> If ${CATEGORY}-${PN}_user and ${CATEGORY}-${PN}_group are set, these
> will replace the contents of USERKIT_USER and USERKIT_GROUP, allowing
> the user to fully define everything user/group related.

How does that all map to multiple users/groups? How does that map to
USE-conditional users/groups? How does that map to users/groups shared
between multiple packages?

Besides, this sounds a lot like games.eclass... will developers be
required to patch upstream software now to force support for using
custom users/groups?

> What happens if the ebuild wants to create multiple users/group?
> Currently, I want to ignore that case and focus on the 80% ebuilds that
> can profit from such an eclass.

Do you have specific numbers? I don't see 80% of ebuilds caring about
users/groups. I don't see the problem you are trying to fix.

Is it one of those problems that someone thinks it's awesome to make
everything declaratory, and add tons of middleware to make the
declaratory work somehow for the most common use cases?

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Manuel Rüger]

[-- Attachment #1.1: Type: text/plain, Size: 3086 bytes --]

On 23.11.2016 10:08, Michał Górny wrote:
> On Wed, 23 Nov 2016 09:44:33 +0100
> Manuel Rüger <mrueg@gentoo.org> wrote:
> 
>> I have not started to write it, but I am considering it and rather want
>> to gather feedback on my idea first.
>> I am aware that https://wiki.gentoo.org/wiki/GLEP:27 exists, but as of
>> right now I haven't seen anyone working on it. The goal of this eclass
>> is to improve user/group handling without touching the PMS.
>>
>> tl;dr: Userkit eclass will improve the user handling by externalizing
>> the configuration to variables that can be set from outside of the ebuild.
>>
>> Userkit.eclass will inherit user.eclass and require bash arrays
>> USERKIT_USER and USERKIT_GROUP for configuration.
>> I will export a pkg_setup with the corresponding setup (basically
>> calling enewuser and enewgroup). It will provide get_user, get_uid,
>> get_group, get_gid and get_home functions.
>> This would allow to do something like "fowners $(get_user):$(get_group)
>> foo".
>>
>> If ${CATEGORY}-${PN}_user and ${CATEGORY}-${PN}_group are set, these
>> will replace the contents of USERKIT_USER and USERKIT_GROUP, allowing
>> the user to fully define everything user/group related.
> 
> How does that all map to multiple users/groups? How does that map to
> USE-conditional users/groups? How does that map to users/groups shared
> between multiple packages?
> 
simply via calling the function conditional in pkg_setup
My goal is not to focus on handling multiple users/groups. Synchronizing
settings between multiple packages is a task of the user, it doesn't
make any sense to make guesses there. People will come up with wonderful
symlinked solutions.
> Besides, this sounds a lot like games.eclass... will developers be
> required to patch upstream software now to force support for using
> custom users/groups?
I am not aware of any patches that are required. What I care about is
having predictable uid/gid and home for everything I can configure via
an ebuild.

> 
>> What happens if the ebuild wants to create multiple users/group?
>> Currently, I want to ignore that case and focus on the 80% ebuilds that
>> can profit from such an eclass.
> 
> Do you have specific numbers? I don't see 80% of ebuilds caring about
> users/groups. I don't see the problem you are trying to fix.
> 
Okay let me rephrase that here: "probably more than 80% of the ebuilds
that are calling enewuser/enewgroup" install a single user or a single
group. There will be some cases this eclass is not applicable to, but
that is fine. If this is something we really want to coveras well using
the eclass based approach, we probably could start enumerating the
variable and if those available to what needs to be done. Something like
USERKIT_USER_2. Not sure if we want to do that.

> Is it one of those problems that someone thinks it's awesome to make
> everything declaratory, and add tons of middleware to make the
> declaratory work somehow for the most common use cases?
> 
I don't see "tons of middleware" here.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 603 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2503 bytes --]

On Wed, 23 Nov 2016 10:19:42 +0100
Manuel Rüger <mrueg@gentoo.org> wrote:

> On 23.11.2016 10:08, Michał Górny wrote:
> > On Wed, 23 Nov 2016 09:44:33 +0100
> > Manuel Rüger <mrueg@gentoo.org> wrote:
> >   
> >> I have not started to write it, but I am considering it and rather want
> >> to gather feedback on my idea first.
> >> I am aware that https://wiki.gentoo.org/wiki/GLEP:27 exists, but as of
> >> right now I haven't seen anyone working on it. The goal of this eclass
> >> is to improve user/group handling without touching the PMS.
> >>
> >> tl;dr: Userkit eclass will improve the user handling by externalizing
> >> the configuration to variables that can be set from outside of the ebuild.
> >>
> >> Userkit.eclass will inherit user.eclass and require bash arrays
> >> USERKIT_USER and USERKIT_GROUP for configuration.
> >> I will export a pkg_setup with the corresponding setup (basically
> >> calling enewuser and enewgroup). It will provide get_user, get_uid,
> >> get_group, get_gid and get_home functions.
> >> This would allow to do something like "fowners $(get_user):$(get_group)
> >> foo".
> >>
> >> If ${CATEGORY}-${PN}_user and ${CATEGORY}-${PN}_group are set, these
> >> will replace the contents of USERKIT_USER and USERKIT_GROUP, allowing
> >> the user to fully define everything user/group related.  
> > 
> > How does that all map to multiple users/groups? How does that map to
> > USE-conditional users/groups? How does that map to users/groups shared
> > between multiple packages?
> >   
> simply via calling the function conditional in pkg_setup
> My goal is not to focus on handling multiple users/groups. Synchronizing
> settings between multiple packages is a task of the user, it doesn't
> make any sense to make guesses there. People will come up with wonderful
> symlinked solutions.
> > Besides, this sounds a lot like games.eclass... will developers be
> > required to patch upstream software now to force support for using
> > custom users/groups?  
> I am not aware of any patches that are required. What I care about is
> having predictable uid/gid and home for everything I can configure via
> an ebuild.

Wait a minute. So you're talking about configuring UID/GID (as
in numbers) and not changing usernames? That's the problem when you
jump straight to solutions and don't state the problem -- nobody knows
what you're really up to.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 3160 bytes --]

On 11/23/2016 01:08 AM, Michał Górny wrote:
> On Wed, 23 Nov 2016 09:44:33 +0100
> Manuel Rüger <mrueg@gentoo.org> wrote:
> 
>> I have not started to write it, but I am considering it and rather want
>> to gather feedback on my idea first.
>> I am aware that https://wiki.gentoo.org/wiki/GLEP:27 exists, but as of
>> right now I haven't seen anyone working on it. The goal of this eclass
>> is to improve user/group handling without touching the PMS.
>>
>> tl;dr: Userkit eclass will improve the user handling by externalizing
>> the configuration to variables that can be set from outside of the ebuild.
>>
>> Userkit.eclass will inherit user.eclass and require bash arrays
>> USERKIT_USER and USERKIT_GROUP for configuration.
>> I will export a pkg_setup with the corresponding setup (basically
>> calling enewuser and enewgroup). It will provide get_user, get_uid,
>> get_group, get_gid and get_home functions.
>> This would allow to do something like "fowners $(get_user):$(get_group)
>> foo".
>>
>> If ${CATEGORY}-${PN}_user and ${CATEGORY}-${PN}_group are set, these
>> will replace the contents of USERKIT_USER and USERKIT_GROUP, allowing
>> the user to fully define everything user/group related.
> 
> How does that all map to multiple users/groups? How does that map to
> USE-conditional users/groups? How does that map to users/groups shared
> between multiple packages?
> 
> Besides, this sounds a lot like games.eclass... will developers be
> required to patch upstream software now to force support for using
> custom users/groups?
> 
>> What happens if the ebuild wants to create multiple users/group?
>> Currently, I want to ignore that case and focus on the 80% ebuilds that
>> can profit from such an eclass.
> 
> Do you have specific numbers? I don't see 80% of ebuilds caring about
> users/groups. I don't see the problem you are trying to fix.
> 
> Is it one of those problems that someone thinks it's awesome to make
> everything declaratory, and add tons of middleware to make the
> declaratory work somehow for the most common use cases?
> 
I think the use-case here is ebuilds that need to create a user and/or
group (www-servers/lighttpd is a good example; alongside pretty much
anything else that needs to run as a separate user and serves
something). In lighttpd's case we don't currently support the ability to
declare which user and group lightty uses; the lighttpd user and
lighttpd group will always be created. Later configuration of users and
groups can be worked with, and iirc we recently patched the initscript
so it handles that use case.

I could see a use-case for someone wanting to install a given daemon or
server with a specific user and/or group. I'm not sure this is the right
approach (nor do I know what is), but I think we have room to think
about a solution; ideally one that is dead-simple to implement and
doesn't have a ton of edge-cases.

What is QA's current policy on user/group creation, btw?
-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1649 bytes --]

On Friday, November 25, 2016 11:39:15 PM EST Daniel Campbell wrote:
>
> I could see a use-case for someone wanting to install a given daemon or
> server with a specific user and/or group. I'm not sure this is the right
> approach (nor do I know what is), but I think we have room to think
> about a solution; ideally one that is dead-simple to implement and
> doesn't have a ton of edge-cases.
> 
> What is QA's current policy on user/group creation, btw?

Years ago there was talk/discussion of having some list/database of UID/GID[1]
[2], so that we have consistent assignment. Arch seems to be the only distro 
thus far who has produced such a list[1], but seems to be outdated and not 
maintained. Also seems to deviate from some UID/GID numbers RedHat uses for 
example[2]. Arch 78 for KVM group, RedHat uses 36.

While there are many reasons people do not care about UID/GID, and arguments 
could be made that it might be better to have them vary on systems and be 
unique. Though some things there are already common UID/GID across distros.

I think in the long run, surely for anyone managing lots of systems. It is far 
better to have a consistent standard list of UID/GID including names. Maybe 
other distro's will adopt and become more of a standard.

1. http://marc.info/?l=gentoo-dev&w=2&r=1&s=Assigning+unique+system+uid%2Fgid
+for+new+&q=b
2. http://marc.info/?t=117034194400005&r=1&w=2
3. https://wiki.archlinux.org/index.php?title=DeveloperWiki:UID_/_GID_Database
4. https://access.redhat.com/documentation/en-US/
Red_Hat_Enterprise_Virtualization/3.5/html/Installation_Guide/sect-
System_Accounts.html

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 3113 bytes --]

On Mon, Nov 28, 2016 at 8:21 AM, William L. Thomson Jr. <wlt-ml@o-sinc.com>
wrote:

> On Friday, November 25, 2016 11:39:15 PM EST Daniel Campbell wrote:
> >
> > I could see a use-case for someone wanting to install a given daemon or
> > server with a specific user and/or group. I'm not sure this is the right
> > approach (nor do I know what is), but I think we have room to think
> > about a solution; ideally one that is dead-simple to implement and
> > doesn't have a ton of edge-cases.
> >
> > What is QA's current policy on user/group creation, btw?
>
> Years ago there was talk/discussion of having some list/database of
> UID/GID[1]
> [2], so that we have consistent assignment. Arch seems to be the only
> distro
> thus far who has produced such a list[1], but seems to be outdated and not
> maintained. Also seems to deviate from some UID/GID numbers RedHat uses for
> example[2]. Arch 78 for KVM group, RedHat uses 36.
>
> While there are many reasons people do not care about UID/GID, and
> arguments
> could be made that it might be better to have them vary on systems and be
> unique. Though some things there are already common UID/GID across distros.
>
> I think in the long run, surely for anyone managing lots of systems. It is
> far
> better to have a consistent standard list of UID/GID including names. Maybe
> other distro's will adopt and become more of a standard.
>

Generally speaking as a fellow who maintained thousands of systems (many of
which ran various operating systems.)

You cannot rely on all OS vendors to synchronize uid / gid. You cannot even
rely on some single vendors to synchronize uid / gids between releases of
their own products. If you build your fleet maintenance with this premise
in mind; most folks I've seen come up with a way to manage it.

Often it means things like:

1) Adding shared accounts to a database and using nsswitch to forward
lookups.
2) Adding configuration management rules to add named accounts to every
machine.
3) Building your fleet such as local uid / gid doesn't matter so much
(often this means the demise of shared filesystems or other bolt-on
authentication / authorization mechanisms.

Typically since most folks building a fleet have to synchronize uid / gid
of actual human users anyway (so people can login / access files / etc) and
so the burden just becomes "give me a list of accounts I should add to my
'syncer' so they are auto-populated on all machines'.

The uids and gids don't matter so much (I can assign them myself, often I
need to inter-operate with other systems where names are already in use,
etc.) But just having a list of "these system accounts are important" is
probably useful on its own.

-A


>
> 1. http://marc.info/?l=gentoo-dev&w=2&r=1&s=Assigning+
> unique+system+uid%2Fgid
> +for+new+&q=b
> 2. http://marc.info/?t=117034194400005&r=1&w=2
> 3. https://wiki.archlinux.org/index.php?title=DeveloperWiki:
> UID_/_GID_Database
> 4. https://access.redhat.com/documentation/en-US/
> Red_Hat_Enterprise_Virtualization/3.5/html/Installation_Guide/sect-
> System_Accounts.html
>
> --
> William L. Thomson Jr.
>

[-- Attachment #2: Type: text/html, Size: 4505 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 899 bytes --]

On Monday, November 28, 2016 10:42:54 AM EST Alec Warner wrote:
>
> Generally speaking as a fellow who maintained thousands of systems (many of
> which ran various operating systems.)
> 
> You cannot rely on all OS vendors to synchronize uid / gid. You cannot even
> rely on some single vendors to synchronize uid / gids between releases of
> their own products.

I believe the main reason such is the case is a lack of any such list or 
database for others to adhere to. Once again an area Gentoo could be leading. 
Had Gentoo done this years ago others might have adopted.

IMHO it is something that should be  a part of LSB. If not POSIX in general. 
One cannot really change the past or current state of things. But can make the 
future better.

For now who cares about other OS or distros. If Gentoo gets its house in order 
others may follow.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[M. J. Everitt]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

On 28/11/16 19:39, William L. Thomson Jr. wrote:
> For now who cares about other OS or distros. If Gentoo gets its house in order 
> others may follow.
>
At the risk of a huge flame, remind me, who uses Gentoo again?!


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 1043 bytes --]

On 11/28/2016 03:26 PM, M. J. Everitt wrote:
> On 28/11/16 19:39, William L. Thomson Jr. wrote:
>> For now who cares about other OS or distros. If Gentoo gets its house in order 
>> others may follow.
>>
> At the risk of a huge flame, remind me, who uses Gentoo again?!
> 
Unless something's changed in the past year or two, iirc Sony uses
Gentoo as part of the backend of Gaikai, Google's used it for the base
of ChromeOS... I can't speak for other 'big names', but Gentoo's not
quite as niche as the small, active userbase has most of us believing.

There's also our downstream neighbors: Funtoo, Pentoo, Sabayon,
Calculate, Exherbo, etc

As for communities, lots of places from 4chan to lainchan, various mesh
network users, security-conscious communities, OCD support groups
(kidding), etc.

I'm sure I'm missing some mentions here; this is just off the top of my
head.
-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 2402 bytes --]

OT, who runs Gentoo

On Saturday, December 3, 2016 12:21:55 AM EST Daniel Campbell wrote:
> 
> Unless something's changed in the past year or two, iirc Sony uses
> Gentoo as part of the backend of Gaikai, Google's used it for the base
> of ChromeOS...

Also is the base of CoreOS, widely popular. Google uses it in more than 
ChromOS it was used in the OnHub router. I suspect Google has wider uses.

Google has hired a few core developers as has Gaikai. Both seem to be good, 
though not sure Google is giving back as much given their financial benefit. 
Gaikai isn't selling an OS, but Google is based on Gentoo...

> I can't speak for other 'big names', but Gentoo's not
> quite as niche as the small, active userbase has most of us believing.

FYI Network Solutions, which I believe at a time had a root DNS server running 
Gentoo. Due to their parent company Web.com is, or has by now, moving from 
Gentoo to RHEL. I was told they do not want to be in the operating system 
development business.

Meetup.com runs Gentoo, not sure how widely but it is in use. Likely more out 
there, but I see far more RHEL and CentOS. I do not even discuss Gentoo, as I 
have to many say it can't be used in production, etc. Things I disagree with.

> There's also our downstream neighbors: Funtoo, Pentoo, Sabayon,
> Calculate, Exherbo, etc

Two of those are more of a splinter of the Gentoo community, Exherbo and 
Funtoo. IMHO those communities should be reunited into Gentoo. Allot of talent 
got driven way that is still out there working on other stuff.

> As for communities, lots of places from 4chan to lainchan, various mesh
> network users, security-conscious communities, OCD support groups
> (kidding), etc.

My concern is lack of any mention in the news or tech articles. When was the 
last time Gentoo was making headlines and not for drama related stuff.

Gentoo News articles
https://www.google.com/search?hl=en&gl=us&tbm=nws&authuser=0&q=gentoo+linux

Arch News articles for example, other distros Ubuntu and Debian have allot
https://www.google.com/search?hl=en&gl=us&tbm=nws&authuser=0&q=arch+linux

Not my words but like say Infoworld articles, not that they are any authority

http://www.infoworld.com/article/3109830/linux/why-did-gentoo-linux-fade-into-obscurity.html

http://www.infoworld.com/article/3137969/linux/arch-linux-the-last-refuge-for-purists.html


-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Rich Freeman]

On Sat, Dec 3, 2016 at 9:00 AM, William L. Thomson Jr.
<wlt-ml@o-sinc.com> wrote:
> OT, who runs Gentoo
> On Saturday, December 3, 2016 12:21:55 AM EST Daniel Campbell wrote:
>
>> There's also our downstream neighbors: Funtoo, Pentoo, Sabayon,
>> Calculate, Exherbo, etc
>
> Two of those are more of a splinter of the Gentoo community, Exherbo and
> Funtoo. IMHO those communities should be reunited into Gentoo. Allot of talent
> got driven way that is still out there working on other stuff.
>

Uh, you do realize that the main force behind one of those projects
left because of the main force behind the other project?

https://lwn.net/Articles/225060/

-- 
Rich

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1647 bytes --]

On Saturday, December 3, 2016 9:09:30 AM EST Rich Freeman wrote:
> On Sat, Dec 3, 2016 at 9:00 AM, William L. Thomson Jr.
> 
> <wlt-ml@o-sinc.com> wrote:
> > OT, who runs Gentoo
> > 
> > On Saturday, December 3, 2016 12:21:55 AM EST Daniel Campbell wrote:
> >> There's also our downstream neighbors: Funtoo, Pentoo, Sabayon,
> >> Calculate, Exherbo, etc
> > 
> > Two of those are more of a splinter of the Gentoo community, Exherbo and
> > Funtoo. IMHO those communities should be reunited into Gentoo. Allot of
> > talent got driven way that is still out there working on other stuff.
> 
> Uh, you do realize that the main force behind one of those projects
> left because of the main force behind the other project?

I think Exherbo was more of a loss, as my old recruiter went there and others 
I had worked with in the past.

> https://lwn.net/Articles/225060/

I hardly recall things as being that simple from that article over banning 
Ciaran. I spoke to Daniel then and since. It was more over not being able to 
lead the project. This was when the foundation needed to be reinstated and 
other things.

Since most those links are broken here is one that is not, I was very close to 
that matter at the time.
https://marc.info/?l=gentoo-dev&m=117303590903513&w=2

Which was one of the last articles Gentoo mentioned in on Distro watch, till I 
believe the OnHub router. Based around that topic, quoting Ciaran.

http://distrowatch.com/weekly.php?issue=20070312#future

Most interesting about that article. If you read the last two paragraphs. I 
think some of that could be said about the state of things still.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Rich Freeman]

On Sat, Dec 3, 2016 at 9:20 AM, William L. Thomson Jr.
<wlt-ml@o-sinc.com> wrote:
>
> Which was one of the last articles Gentoo mentioned in on Distro watch, till I
> believe the OnHub router. Based around that topic, quoting Ciaran.
>
> http://distrowatch.com/weekly.php?issue=20070312#future
>
> Most interesting about that article. If you read the last two paragraphs. I
> think some of that could be said about the state of things still.
>

Sure, and it probably will be the state of things 20 years from now,
with Gentoo still having "little chance that even the minimum of
release and bug-fixing goals will be met" and suffering a "rapid
downfall of the distribution" :)

The predictions of those paragraphs have not in fact come to pass.

Would you agree that "if a person who repeatedly engages in personal
attacks against other developers is permitted to remain with the
project, then there is something wrong with the way the distribution
is managed?"

I find it a bit interesting that half of this article is about a
failure to enforce a Code of Conduct that you don't actually think we
ought to have, and that drobbins left in part because it wasn't being
enforced.

Sometimes forks exist because individuals don't get along or have
strong ideas for how things should work to the exclusion of other
ideas of how things should work.  That's fine, there is nothing wrong
with forks.

The current meta-structure of Gentoo is structured around the vision
that Gentoo is a place where people can make what they want of it, and
the governance bodies of Gentoo are mostly about dealing with
conflicting goals, not picking winners.  Sure, the Council could take
a much more pro-active stance and say "Gentoo needs to be the best
distro for xyz so we should get rid of all this Java crap" but that
would be silly because the two aren't mutually exclusive and telling
people to not work on Java isn't going to magically inspire them to
work on something else instead.

-- 
Rich

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 2121 bytes --]

Getting further off topic, rather not create noise to bother others, minimal 
below.

On Saturday, December 3, 2016 9:36:47 AM EST Rich Freeman wrote:
> 
> Sure, and it probably will be the state of things 20 years from now,
> with Gentoo still having "little chance that even the minimum of
> release and bug-fixing goals will be met" and suffering a "rapid
> downfall of the distribution" :)
>
> The predictions of those paragraphs have not in fact come to pass.

The article is no spot on to things today, but does have lots of similarities.

> Would you agree that "if a person who repeatedly engages in personal
> attacks against other developers is permitted to remain with the
> project, then there is something wrong with the way the distribution
> is managed?"

I do not see how Gentoo the project as a whole is managed

> I find it a bit interesting that half of this article is about a
> failure to enforce a Code of Conduct that you don't actually think we
> ought to have, and that drobbins left in part because it wasn't being
> enforced.

That article is not correct on the Daniel Robbins aspect. I can have Daniel 
comment if you like. It had more to do with leading Gentoo, resuming his 
previous role etc. Nothing relating to CoC or individuals.

> Sometimes forks exist because individuals don't get along or have
> strong ideas for how things should work to the exclusion of other
> ideas of how things should work.  That's fine, there is nothing wrong
> with forks.

It does not help the main community. My favorite story of such is Firebird and 
its Vulcan fork. Which long story short was merged back into firebird and 
became Firebird 3 :)

Good stuff can happen when people reunite. Not always the case with forks. 
Some forks die. Not implying that in either case but historically that is the 
case.  Like XFree86 for example, over license changes.

> The current meta-structure of Gentoo is structured around the vision

https://archives.gentoo.org/gentoo-project/message/
3ac5418dd061fc53f4b8d55a99773f4c

Been here before, said it all before... No need to repeat.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Tobias Klausmann]

Hi! 

On Sat, 03 Dec 2016, William L. Thomson Jr. wrote:
> Google has hired a few core developers as has Gaikai. Both seem
> to be good, though not sure Google is giving back as much given
> their financial benefit. Gaikai isn't selling an OS, but Google
> is based on Gentoo...

That last bit is not true. While yes, Chrome OS and Core OS had a
Gentoo base, a lot was done of top of that.

Furthermore, most of Google's products (both shipped devices and
services) are _not_ based on Gentoo and never were. The base of
Google's production server OS is another Linux distribution, and
there, too, a lot of what makes the thing tick has no outside
equivalent.

Source: I worked on the relevant Google production teams for six
years.


Regards,
Tobias



-- 
panic("Foooooooood fight!");
        linux-2.2.16/drivers/scsi/aha1542.c

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

On Saturday, December 3, 2016 5:58:29 PM EST Tobias Klausmann wrote:
> Hi!
> 
> On Sat, 03 Dec 2016, William L. Thomson Jr. wrote:
> > Google has hired a few core developers as has Gaikai. Both seem
> > to be good, though not sure Google is giving back as much given
> > their financial benefit. Gaikai isn't selling an OS, but Google
> > is based on Gentoo...
> 
> That last bit is not true. While yes, Chrome OS and Core OS had a
> Gentoo base, a lot was done of top of that.

I realize I left out ChromeOS in the previous comment, last bit. I was not 
meaning to imply Google ran on Gentoo, just ChromeOS was based on it.

Thanks for some insight into Google though!

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]

On Sat, 03 Dec 2016 12:08:44 -0500
"William L. Thomson Jr." <wlt-ml@o-sinc.com> wrote:

> On Saturday, December 3, 2016 5:58:29 PM EST Tobias Klausmann wrote:
> > Hi!
> > 
> > On Sat, 03 Dec 2016, William L. Thomson Jr. wrote:  
> > > Google has hired a few core developers as has Gaikai. Both seem
> > > to be good, though not sure Google is giving back as much given
> > > their financial benefit. Gaikai isn't selling an OS, but Google
> > > is based on Gentoo...  
> > 
> > That last bit is not true. While yes, Chrome OS and Core OS had a
> > Gentoo base, a lot was done of top of that.  
> 
> I realize I left out ChromeOS in the previous comment, last bit. I
> was not meaning to imply Google ran on Gentoo, just ChromeOS was
> based on it.
> 
> Thanks for some insight into Google though!
> 

I think you will find that ChromeOS is not really based on Gentoo, but
built using Gentoo as the base os it IS built from.  It is still Linux
based, but ChromeOS doe not use ebuilds and such for normal
operations.  But the binary pkgs that build it are custom built using
Gentoo's ebuild system and modified package management systems.  The
final resulting OS has it's own methods of updating itself and
installing the binaries.  (At least that is the way it was some years
ago, last I had looked)

-- 
Brian Dolbec <dolsen>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1241 bytes --]

On Saturday, December 3, 2016 11:27:20 AM EST Brian Dolbec wrote:
>
> I think you will find that ChromeOS is not really based on Gentoo, but
> built using Gentoo as the base os it IS built from.  It is still Linux
> based, but ChromeOS doe not use ebuilds and such for normal
> operations.  But the binary pkgs that build it are custom built using
> Gentoo's ebuild system and modified package management systems.

All that sounds like it is based on Gentoo. Its packages before they turn them 
into binaries are ebuilds.

> The
> final resulting OS has it's own methods of updating itself and
> installing the binaries.  (At least that is the way it was some years
> ago, last I had looked)

Yes, but if you want to modify any binaries etc, you are back to ebuilds. If 
you want to add a new package not available its the same thing. They use 
overlay concepts etc.

I do get ChromeOS is very different, kinda like Android, you must flash images 
you build. No installing it in the normal sense.

No matter what terms you use, Google uses Gentoo to build products it makes 
money off. In that sense I think it could give back allot. If not in 
donations, resources (tinderbox), etc.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Brian Dolbec]

[-- Attachment #1: Type: text/plain, Size: 1764 bytes --]

On Sat, 03 Dec 2016 16:49:19 -0500
"William L. Thomson Jr." <wlt-ml@o-sinc.com> wrote:

> On Saturday, December 3, 2016 11:27:20 AM EST Brian Dolbec wrote:
> >
> > I think you will find that ChromeOS is not really based on Gentoo,
> > but built using Gentoo as the base os it IS built from.  It is
> > still Linux based, but ChromeOS doe not use ebuilds and such for
> > normal operations.  But the binary pkgs that build it are custom
> > built using Gentoo's ebuild system and modified package management
> > systems.  
> 
> All that sounds like it is based on Gentoo. Its packages before they
> turn them into binaries are ebuilds.
> 
> > The
> > final resulting OS has it's own methods of updating itself and
> > installing the binaries.  (At least that is the way it was some
> > years ago, last I had looked)  
> 
> Yes, but if you want to modify any binaries etc, you are back to
> ebuilds. If you want to add a new package not available its the same
> thing. They use overlay concepts etc.
> 
> I do get ChromeOS is very different, kinda like Android, you must
> flash images you build. No installing it in the normal sense.
> 
> No matter what terms you use, Google uses Gentoo to build products it
> makes money off. In that sense I think it could give back allot. If
> not in donations, resources (tinderbox), etc.
> 

Well, It does give back some.  I know there are times patches are
pushed to gentoo from them.  Plus we get financial support from them
via the Google Summer Of Code.  It has been pointed out recently that
the majority of our available funds have come from the GSOC.  Not only
that, but it also helps us get new developers to work with and further
Gentoo projects.

-- 
Brian Dolbec <dolsen>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1649 bytes --]

On Saturday, December 3, 2016 1:56:40 PM EST Brian Dolbec wrote:
>
> Well, It does give back some.  I know there are times patches are
> pushed to gentoo from them.  

Yes, and they employ a couple current and former devs. At least one quite core

> Plus we get financial support from them
> via the Google Summer Of Code.  It has been pointed out recently that
> the majority of our available funds have come from the GSOC.  Not only
> that, but it also helps us get new developers to work with and further
> Gentoo projects.

GSoC is controversial. They did not do it for Gentoo 1 year, I think 2015. 
Also it has lead to a bunch of projects started that are not used. One I am 
not sure why is not used, euscan, as its highly beneficial. Others seemed to 
have fallen off rather than become integrated into Gentoo or lived on as 
projects of their own. I could be wrong on a few, but last I checked that was 
the case for most.

As for the main financial resource for Gentoo that is another thing semi off 
topic for this list. I may have brought it up on -nfp years ago. In a nutshell 
unless Gentoo spends money, and has plans to use money to further development. 
Which in turn benefits others. There is not much motivation for people to 
donate to Gentoo.

There is also the charitable donation and write off aspect. Which they may be 
able to do. But since Gentoo has never received official 501c6 status or any 
from the IRS. I am not sure if companies or anyone can actually write off a 
Gentoo donation. May be moot for individuals, but not for large businesses 
with stringent accounts and standards to meet.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Rich Freeman]

On Sat, Dec 3, 2016 at 5:09 PM, William L. Thomson Jr.
<wlt-ml@o-sinc.com> wrote:
> There is also the charitable donation and write off aspect. Which they may be
> able to do. But since Gentoo has never received official 501c6 status or any
> from the IRS. I am not sure if companies or anyone can actually write off a
> Gentoo donation. May be moot for individuals, but not for large businesses
> with stringent accounts and standards to meet.
>

Actually, it is the opposite.  When you spend money as an individual
in the US it normally doesn't lower your taxes unless there is some
special tax deduction for it, such as the mortgage interest deduction.
One of those deductions is donations to 501c3/etc status.  So, there
is a benefit to an individual when donating to a 501c3 organization
(or other deductible classes) because it lowers their tax burden.

On the other hand, businesses are only taxed on their profits at the
federal level.  So, if a business takes in $500 and spends $400 then
it is taxed on $100.  That $400 could be spent on almost anything as
far as I'm aware.

So, money given to Gentoo by a private business is the same as money
spent on toilet paper or money thrown in the furnace as far as tax
liability goes.  It increases expenses which means it decreases
profits.

Now, where 501c3/etc status does start mattering for businesses is
internal compliance controls.  Most publicly traded companies have
standards for how money can be spent, because that money belongs to
the shareholders.  I work for a publicly traded company and I can't
just treat myself to a new car and expense it, because that deprives
the shareholders of their profits, even if the US government wouldn't
otherwise have a problem with it from a tax perspective (as long as I
declare the value of that car on my own taxes as income).  In order to
keep things simple companies often use 501c3 status as a requirement
for donations.  This eliminates debates about whether a particular
cause is or isn't a valid charity to donate to for the purposes of
goodwill/etc because the IRS acts as an unbiased filter.  501c3 also
implies financial controls on how the money gets spent, so there is
less of a risk that somebody is directing money towards a recipient
who ultimately is going to offer some kind of kickback, because that
would be illegal for the 501c3 and the IRS would enforce that (from a
tax perspective the kickback probably isn't illegal for the original
donor company, but from a shareholder responsibility standpoint it is
a misuse of funds for employees to basically be giving money to
themselves).

So, if your goal is to be the beneficiary of corporate philanthropy,
then I'm sure 501c3 status will help.

Another source of donations might be other 501c3 foundations.  The FSF
might give money to a FOSS-only linux distro, for example.  In such
situations they're almost always going to donate purely to other 501c3
organizations, because they need to ensure the money is spent on
charitable purposes to meet their own IRS requirements.

Now, companies probably also make investments that aren't intended to
be philanthropic.  A company might give money to a trade association
in exchange for some kind of benefit, or it might just give money to
an association to support their which which somehow benefits the
company.  I suspect a business that benefits from Gentoo more directly
probably wouldn't care so much about the tax-exempt status because the
donations are being justified on the basis of being a business
investment of sorts.

So, yes, the status matters, but not actually for tax reasons
themselves in most cases.  It is more of a marker of how the money
gets spent.

I used the term 501c3 just to keep this simple, but there are other
classifications in the tax code which could also apply to an
organization like Gentoo and generally be treated similarly.

If somebody is a corporate tax accountant and wants to offer a finer
explanation it is welcome, but this is the gist of it as I understand
things.

-- 
Rich

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1061 bytes --]

WAYYYY OT :(

On Saturday, December 3, 2016 5:34:56 PM EST Rich Freeman wrote:
> On Sat, Dec 3, 2016 at 5:09 PM, William L. Thomson Jr.
>
> Actually, it is the opposite.  When you spend money as an individual
> in the US it normally doesn't lower your taxes unless there is some
> special tax deduction for it, such as the mortgage interest deduction.
> One of those deductions is donations to 501c3/etc status.  So, there
> is a benefit to an individual when donating to a 501c3 organization
> (or other deductible classes) because it lowers their tax burden.

Gentoo has no official status with the IRS, 501c6, or 501c3, etc. Any write off 
would have to be categorized otherwise.

> So, money given to Gentoo by a private business is the same as money
> spent on toilet paper or money thrown in the furnace as far as tax
> liability goes.  It increases expenses which means it decreases
> profits.

It depends on how a business or anyone wants to categorize. You can only 
deduct so much in any category.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 2427 bytes --]

On Sat, Dec 03, 2016 at 04:49:19PM -0500, William L. Thomson Jr. wrote:
> No matter what terms you use, Google uses Gentoo to build products it makes 
> money off. In that sense I think it could give back allot. If not in 
> donations, resources (tinderbox), etc.
Here's the financial parts of what Google has given us, that I can
quantify. It DOESN'T include anything of the 20% time that might have
been used in Gentoo's favour [some past Google-employed developers have
specifically said they were spending Friday afternoons doing Gentoo
dev], or paid Gentoo stuff that overlapped their actual job needs
(ChromeOS-related).

Non-GSOC:
---------
2011: Google donated brand-new Dell servers, that with their volume
discount, had an invoice price of $4331.60, and a Dell list price over
$6k. Those servers are hosted at OSUOSL, and are still in active use
(dipper, blackcap). 
Net Subtotal: $4,331.60

GSOC:
-----
Gross Payments
(including reimbursement for mentor summit travel expenses)
2009:  $5,151.59
2006:  $7,000.00
2007:  $4,500.00
2008:  $3,000.00
2010: $11,001.25
2011:  $9,891.77
2012:  $7,000.00
2013:      $0.00 [1]
2014:  $4,200.00
2015:      $0.00 - did not participate
2016:  $4,700.00 [2] 
Gross Subtotal: $56,444.61
Less reimbursed travel expenses: $9,852.02
Net Subtotal: $46,592.59

----------------
Net Total: $50,924.19
----------------

[1] 
- Details in bug 488142. Not locked, no personal information in it
  (other year GSOC bugs are locked due to containing mentor personal
  information).
- No record could be found for a 2013 invoice, in our bank records OR
  Google's invoice archive [MANY thanks to Antarus and the Google
  accounting department who did pull all historical invoices submitted
  by Gentoo].
- presumed we did not submit before the deadline and thus forfeited the
  payment & reimbursement.
- Net Amount if we had filed: $2591.55
-- 6 students * $500/student: $3000
-- Less $2608.45 in actual travel expenses (max $2200 reimbursement)
- Net for non-filing: $-2608.45
- NOT included in reimbursement subtotal above.

[2] Invoice submitted 2012/11/29, payment NET30 period ends 2016/12/29

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 1083 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1722 bytes --]

OT SNR Sorry but not sorry :) 

On Saturday, December 3, 2016 11:01:56 PM EST Robin H. Johnson wrote:
> On Sat, Dec 03, 2016 at 04:49:19PM -0500, William L. Thomson Jr. wrote:
> > No matter what terms you use, Google uses Gentoo to build products it
> > makes
> > money off. In that sense I think it could give back allot. If not in
> > donations, resources (tinderbox), etc.
> 
> Here's the financial parts of what Google has given us, that I can
> quantify. It DOESN'T include anything of the 20% time that might have
> been used in Gentoo's favour [some past Google-employed developers have
> specifically said they were spending Friday afternoons doing Gentoo
> dev], or paid Gentoo stuff that overlapped their actual job needs
> (ChromeOS-related).

Not sure how to measure time spent on job hours in contribution dollars.

> ----------------
> Net Total: $50,924.19
> ----------------

So from 09-16 avg of ~$4.6k per year over 11 years. With that really being 
earned by people doing GSoC. Not the same as if Google donated a lump sum of 
money to further development per say the Councils plans. Only 1 hardware 
donation.

I believe past sponsors such as GNi incurred costs in the ~$5k range monthly. 
I would assume some hosting sponsors to be averaging a few thousand at minimum 
per year.

Just as an example. FreeBSD is seeking $1.25 Million in a fundraiser with 
$882k thus far.
https://www.freebsdfoundation.org/

They seem to average in the hundreds of thousands every year in contributions
https://www.freebsdfoundation.org/about/financials/

Always looked at FreeBSD when I was a Gentoo Trustee. Great foundation! Passed 
the 5 year probation period with IRS, and other stuff.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Robin H. Johnson]

On Sat, Dec 03, 2016 at 06:30:29PM -0500, William L. Thomson Jr. wrote:
> > ----------------
> > Net Total: $50,924.19
> > ----------------
> So from 09-16 avg of ~$4.6k per year over 11 years. 
10 years of participation, 9 of which we got paid for. So ~$5.7k/year.
If we got paid for 2013: ~$5.4k/year over 10 years.

> With that really being earned by people doing GSoC. Not the same as if
> Google donated a lump sum of money to further development per say the
> Councils plans. Only 1 hardware donation.
That's the payment to the organization for mentoring and managing the
students, separate from what the students doing GSoC earned.

If the student's work was of use to Gentoo, then it's ALSO $5000-$5500
per student that we've had in man-hours. I do use that disclaimer,
because I know the integration rate for Gentoo students much lower than
it should be.

2006: 10 students
2007: 8 students
2008: 5 students
2009: 6 students
2010: 16 students
2011: 14 students
2012: 8 students
2013: 6 students
2014: 3 students
2016: 5 students

Total: 81 students.
Assuming $5k/student: $405,000 in student payments, over 11 years.

I don't know how many students we've failed: I do know it's been at
least one (I failed them. Their original mentor had medical issues, I
took over, and they provided a mocked video of their work and no code by
midterm).

> I believe past sponsors such as GNi incurred costs in the ~$5k range monthly. 
> I would assume some hosting sponsors to be averaging a few thousand at minimum 
> per year.
The cost to GNi was much closer to $1k/month, mostly in potential lost
revenue if the hardware COULD be used for income (it was already a sunk
cost, and didn't have other users). For our present major hosting
sponsors, I believe we're more in line with $250-$400/month, but again
mostly older hardware that isn't of much other salable use.

> Just as an example. FreeBSD is seeking $1.25 Million in a fundraiser with 
> $882k thus far.
> https://www.freebsdfoundation.org/
$1.25M is their annual fund-raising target for this year and last. Not a
specific fund-raiser, but their annual target.
For 2016 Q1-Q3, on the $1.25M, they report $293k in contributions.
For 2015, on a $1.25M target, they reported $657k in contributions.
For 2014, on a $1M target, they reported $2.4M in contributions.

> They seem to average in the hundreds of thousands every year in contributions
> https://www.freebsdfoundation.org/about/financials/
They're also got a good few years on us (as do Apache).

> Always looked at FreeBSD when I was a Gentoo Trustee. Great foundation! Passed 
> the 5 year probation period with IRS, and other stuff.
The Apache Foundation was very beneficial to look at I found, because
they kept superb public records, but also were not hampered by some of
our restrictions about depending on non-open software (they & the perl
foundation BOTH use QuickBooks on Windows for their accounting).

https://www.apache.org/foundation/records/

I draw your attention to their last 990 filing:
https://www.apache.org/foundation/records/990-2014.pdf
- $1.2M in annual income
- $858k spend on infrastructure, 
  of which >$400k was marked directly as IT spending.
- $1.8M in net assets

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 4638 bytes --]

On Sunday, December 4, 2016 7:22:51 AM EST Robin H. Johnson wrote:
> On Sat, Dec 03, 2016 at 06:30:29PM -0500, William L. Thomson Jr. wrote:
> > > ----------------
> > > Net Total: $50,924.19
> > > ----------------
> > 
> > So from 09-16 avg of ~$4.6k per year over 11 years.
> 
> 10 years of participation, 9 of which we got paid for. So ~$5.7k/year.
> If we got paid for 2013: ~$5.4k/year over 10 years.

I still think it is relatively low.

> > With that really being earned by people doing GSoC. Not the same as if
> > Google donated a lump sum of money to further development per say the
> > Councils plans. Only 1 hardware donation.
> 
> That's the payment to the organization for mentoring and managing the
> students, separate from what the students doing GSoC earned.
> 
> If the student's work was of use to Gentoo, then it's ALSO $5000-$5500
> per student that we've had in man-hours. I do use that disclaimer,
> because I know the integration rate for Gentoo students much lower than
> it should be.

My only point was without someone doing work. Google does not give Gentoo the 
money. It is not like a normal person or business donating to or sponsoring 
Gentoo. That is their specialized program. I have to think they have their own 
interest in it as well.

I would almost say the donation from both the person in time and google is 
from the person not Google. Also including the mentor or what ever that is 
called. Those 2 people put in their time, to make sure that money is paid to 
the student and organization.

> The cost to GNi was much closer to $1k/month, mostly in potential lost
> revenue if the hardware COULD be used for income (it was already a sunk
> cost, and didn't have other users). For our present major hosting
> sponsors, I believe we're more in line with $250-$400/month, but again
> mostly older hardware that isn't of much other salable use.

That would ruffly put GNi ~$12k a year. How many hosting sponsors?
Even on the low side $250 x 12 = $3k, $400 is $4800. I doubt any have the 
revenue of Google. Or shipping products like ChromeOS or OnHub router using 
Gentoo build system.

I just do not see GSoC in the same light as others and over all from Google is 
pretty minimal IMHO. Relative to their benefit, revenue derived from such, and 
over all revenue of the company.

They could provide cloud resources and other that would likely not cost them 
much in overhead.

> > Just as an example. FreeBSD is seeking $1.25 Million in a fundraiser with
> > $882k thus far.
> > https://www.freebsdfoundation.org/
> 
> $1.25M is their annual fund-raising target for this year and last. Not a
> specific fund-raiser, but their annual target.
> For 2016 Q1-Q3, on the $1.25M, they report $293k in contributions.
> For 2015, on a $1.25M target, they reported $657k in contributions.
> For 2014, on a $1M target, they reported $2.4M in contributions.

Still far beyond Gentoo.

> > They seem to average in the hundreds of thousands every year in
> > contributions https://www.freebsdfoundation.org/about/financials/
> 
> They're also got a good few years on us (as do Apache).

Yes for sure, but Gentoo has set itself back needlessly. Things that should 
have been done haven't. Much less other things that could have happened. 
Funding for events per se, Gentoo Conference....

> > Always looked at FreeBSD when I was a Gentoo Trustee. Great foundation!
> > Passed the 5 year probation period with IRS, and other stuff.
> 
> The Apache Foundation was very beneficial to look at I found, because
> they kept superb public records, but also were not hampered by some of
> our restrictions about depending on non-open software (they & the perl
> foundation BOTH use QuickBooks on Windows for their accounting).
> 
> https://www.apache.org/foundation/records/
> 
> I draw your attention to their last 990 filing:
> https://www.apache.org/foundation/records/990-2014.pdf
> - $1.2M in annual income
> - $858k spend on infrastructure,
>   of which >$400k was marked directly as IT spending.
> - $1.8M in net assets

See they are spending the money. That is something I proposed long ago. To pay 
some hosting sponsors, rather than rely on free hosting sponsors. Which at the 
time were given ad space on g.o. That has since changed, but still goes back 
to if you do not have a plan to spend and use the money to further. Not likely 
to get it.

Sure not all reach their fund raising goals, but at least making such efforts. 
No clue what would happen if Gentoo set out to raise funds. But without plans 
to use, not sure what Gentoo would do with any raised money or donations.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[james]

On 12/04/2016 02:22 AM, Robin H. Johnson wrote:
> On Sat, Dec 03, 2016 at 06:30:29PM -0500, William L. Thomson Jr. wrote:
>>> ----------------
>>> Net Total: $50,924.19
>>> ----------------
>> So from 09-16 avg of ~$4.6k per year over 11 years.
> 10 years of participation, 9 of which we got paid for. So ~$5.7k/year.
> If we got paid for 2013: ~$5.4k/year over 10 years.
>
>> With that really being earned by people doing GSoC. Not the same as if
>> Google donated a lump sum of money to further development per say the
>> Councils plans. Only 1 hardware donation.
> That's the payment to the organization for mentoring and managing the
> students, separate from what the students doing GSoC earned.
>
> If the student's work was of use to Gentoo, then it's ALSO $5000-$5500
> per student that we've had in man-hours. I do use that disclaimer,
> because I know the integration rate for Gentoo students much lower than
> it should be.
>
> 2006: 10 students
> 2007: 8 students
> 2008: 5 students
> 2009: 6 students
> 2010: 16 students
> 2011: 14 students
> 2012: 8 students
> 2013: 6 students
> 2014: 3 students
> 2016: 5 students
>
> Total: 81 students.
> Assuming $5k/student: $405,000 in student payments, over 11 years.
>
> I don't know how many students we've failed: I do know it's been at
> least one (I failed them. Their original mentor had medical issues, I
> took over, and they provided a mocked video of their work and no code by
> midterm).
>
>> I believe past sponsors such as GNi incurred costs in the ~$5k range monthly.
>> I would assume some hosting sponsors to be averaging a few thousand at minimum
>> per year.
> The cost to GNi was much closer to $1k/month, mostly in potential lost
> revenue if the hardware COULD be used for income (it was already a sunk
> cost, and didn't have other users). For our present major hosting
> sponsors, I believe we're more in line with $250-$400/month, but again
> mostly older hardware that isn't of much other salable use.
>
>> Just as an example. FreeBSD is seeking $1.25 Million in a fundraiser with
>> $882k thus far.
>> https://www.freebsdfoundation.org/
> $1.25M is their annual fund-raising target for this year and last. Not a
> specific fund-raiser, but their annual target.
> For 2016 Q1-Q3, on the $1.25M, they report $293k in contributions.
> For 2015, on a $1.25M target, they reported $657k in contributions.
> For 2014, on a $1M target, they reported $2.4M in contributions.
>
>> They seem to average in the hundreds of thousands every year in contributions
>> https://www.freebsdfoundation.org/about/financials/
> They're also got a good few years on us (as do Apache).
>
>> Always looked at FreeBSD when I was a Gentoo Trustee. Great foundation! Passed
>> the 5 year probation period with IRS, and other stuff.
> The Apache Foundation was very beneficial to look at I found, because
> they kept superb public records, but also were not hampered by some of
> our restrictions about depending on non-open software (they & the perl
> foundation BOTH use QuickBooks on Windows for their accounting).


GNUcash is superior to Quickbooks, as it is a 'double entry' accounting 
system. Last time I check Quickbooks was not 'double entry' and that is 
a big deal in accounting.  There is a module that allows entries via 
Android now with GNUcash, but is not an official part of GNUcash.org. I 
use GNUcash with my company, but not the Android smartphone module.....


http://gnucash.org

http://www.techrepublic.com/article/gnucash-a-powerful-mobile-financial-tool-for-android/


Serious inquires could be directed to 'gnucash-user@gnucash.org' as this 
accounting software is robust, under active development and even the 
devs 'chime in' on  routine basis.  All in all, gnucash is an 
outstanding piece of FOSS software; much better than Quickbooks as many 
on the discussion lists attest to on a routine basis. It is in portage 
and it runs on windows and other platforms.


hth,
James


> https://www.apache.org/foundation/records/
>
> I draw your attention to their last 990 filing:
> https://www.apache.org/foundation/records/990-2014.pdf
> - $1.2M in annual income
> - $858k spend on infrastructure,
>   of which >$400k was marked directly as IT spending.
> - $1.8M in net assets
>

[gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[Robin H. Johnson]

(OT accounting systems)

On Sun, Dec 04, 2016 at 01:10:16PM -0500, james wrote:
> GNUcash is superior to Quickbooks, as it is a 'double entry' accounting 
> system. Last time I check Quickbooks was not 'double entry' and that is 
> a big deal in accounting. 
QuickBooks is double-entry, and has been for a very long time; It did
used to obscure the fact before, to make accounting 'easier' for
non-accounting people to understand.

For the Foundation, I'm presently using Ledger-CLI [1], but this is a
hurdle for any third-party financial auditing (we should be prepared at
all times for a real financial audit), because they want data in
quickbooks format.

If there is a good GNUCash support for non-profit accounting (which does
differ from small-business accounting, see [2]), and matching
documentation for it, I'm VERY interested to know about it.

Why Ledger? The Software Freedom Conservancy started a project aimed at
Non-Profit accounting [3], wrapped around Ledger, which covers far more
of the non-profit nuances than GNUCash does.

They included enough documentation in how to specifically configure
Ledger for non-profit usage, so it was easy to get going since I already
used Ledger for my personal accounting.

Ledger being plain-text based does work very well with version control,
even for multiple parties (I enlisted help to convert old bank
statements).

[1] http://www.ledger-cli.org/
[2] http://www.accountingcoach.com/nonprofit-accounting/explanation/1
[3] https://sfconservancy.org/npoacct/

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[M. J. Everitt]

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

On 04/12/16 22:55, Robin H. Johnson wrote:
> (OT accounting systems)
>
> On Sun, Dec 04, 2016 at 01:10:16PM -0500, james wrote:
>> GNUcash is superior to Quickbooks, as it is a 'double entry' accounting 
>> system. Last time I check Quickbooks was not 'double entry' and that is 
>> a big deal in accounting. 
> QuickBooks is double-entry, and has been for a very long time; It did
> used to obscure the fact before, to make accounting 'easier' for
> non-accounting people to understand.
>
> For the Foundation, I'm presently using Ledger-CLI [1], but this is a
> hurdle for any third-party financial auditing (we should be prepared at
> all times for a real financial audit), because they want data in
> quickbooks format.
>
> If there is a good GNUCash support for non-profit accounting (which does
> differ from small-business accounting, see [2]), and matching
> documentation for it, I'm VERY interested to know about it.
>
> Why Ledger? The Software Freedom Conservancy started a project aimed at
> Non-Profit accounting [3], wrapped around Ledger, which covers far more
> of the non-profit nuances than GNUCash does.
>
> They included enough documentation in how to specifically configure
> Ledger for non-profit usage, so it was easy to get going since I already
> used Ledger for my personal accounting.
>
> Ledger being plain-text based does work very well with version control,
> even for multiple parties (I enlisted help to convert old bank
> statements).
>
> [1] http://www.ledger-cli.org/
> [2] http://www.accountingcoach.com/nonprofit-accounting/explanation/1
> [3] https://sfconservancy.org/npoacct/
>
I gather both Quickbooks and Sage have a more modular approach to
"proper" accounting software applicable to small and large businesses. I
know my mother used Quickbooks in the past with good success and the
support of her accountant, but Sage is known to be equally accessible. I
would imagine there is an appropriate version for not-for-profit or
charities, perhaps you can seek advice with the person(s) already
contacted for accounting/finance purposes?!


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[Robin H. Johnson]

On Sun, Dec 04, 2016 at 11:07:59PM +0000, M. J. Everitt wrote:
> I gather both Quickbooks and Sage have a more modular approach to
> "proper" accounting software applicable to small and large businesses. I
> know my mother used Quickbooks in the past with good success and the
> support of her accountant, but Sage is known to be equally accessible. I
> would imagine there is an appropriate version for not-for-profit or
> charities, perhaps you can seek advice with the person(s) already
> contacted for accounting/finance purposes?!
Our CPA (Yes, we do have one) only recommends QuickBooks, but has used a
variety of other proprietary systems (none of which he recommends at
all!).

The catch is that either Quickbooks or Sage would be a violation of the
social contract's libre-licence dependence clause.

Ledger HAS filled most of our needs thus far, but lacks in reporting and
some automation:
- I'd love to automatically generate lots of depreciation
  entries, but can't yet.
- Something to anonymize private information in some entries, so that
  the actual Ledgers can be published for transparency.

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[M. J. Everitt]

[-- Attachment #1: Type: text/plain, Size: 1357 bytes --]

On 04/12/16 23:49, Robin H. Johnson wrote:
> On Sun, Dec 04, 2016 at 11:07:59PM +0000, M. J. Everitt wrote:
>> I gather both Quickbooks and Sage have a more modular approach to
>> "proper" accounting software applicable to small and large businesses. I
>> know my mother used Quickbooks in the past with good success and the
>> support of her accountant, but Sage is known to be equally accessible. I
>> would imagine there is an appropriate version for not-for-profit or
>> charities, perhaps you can seek advice with the person(s) already
>> contacted for accounting/finance purposes?!
> Our CPA (Yes, we do have one) only recommends QuickBooks, but has used a
> variety of other proprietary systems (none of which he recommends at
> all!).
>
> The catch is that either Quickbooks or Sage would be a violation of the
> social contract's libre-licence dependence clause.
>
> Ledger HAS filled most of our needs thus far, but lacks in reporting and
> some automation:
> - I'd love to automatically generate lots of depreciation
>   entries, but can't yet.
> - Something to anonymize private information in some entries, so that
>   the actual Ledgers can be published for transparency.
>
Thanks for the clarification, Robin. It may be worth reviewing that
social contract to allow us better compliance if deemed worthwhile!

:]


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 1848 bytes --]

On 12/04/2016 04:03 PM, M. J. Everitt wrote:
> On 04/12/16 23:49, Robin H. Johnson wrote:
>> On Sun, Dec 04, 2016 at 11:07:59PM +0000, M. J. Everitt wrote:
>>> I gather both Quickbooks and Sage have a more modular approach to
>>> "proper" accounting software applicable to small and large businesses. I
>>> know my mother used Quickbooks in the past with good success and the
>>> support of her accountant, but Sage is known to be equally accessible. I
>>> would imagine there is an appropriate version for not-for-profit or
>>> charities, perhaps you can seek advice with the person(s) already
>>> contacted for accounting/finance purposes?!
>> Our CPA (Yes, we do have one) only recommends QuickBooks, but has used a
>> variety of other proprietary systems (none of which he recommends at
>> all!).
>>
>> The catch is that either Quickbooks or Sage would be a violation of the
>> social contract's libre-licence dependence clause.
>>
>> Ledger HAS filled most of our needs thus far, but lacks in reporting and
>> some automation:
>> - I'd love to automatically generate lots of depreciation
>>   entries, but can't yet.
>> - Something to anonymize private information in some entries, so that
>>   the actual Ledgers can be published for transparency.
>>
> Thanks for the clarification, Robin. It may be worth reviewing that
> social contract to allow us better compliance if deemed worthwhile!
> 
> :]
> 
Compliance with what? If others desire Quickbook support, they can make
a tool to convert from ledger. There's no good reason for a non-profit,
libre software organization to use and depend on proprietary software.
Did nobody learn a lesson from BitKeeper?

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[Ciaran McCreesh]

On Sun, 4 Dec 2016 18:47:48 -0800
Daniel Campbell <zlg@gentoo.org> wrote:
> Compliance with what? If others desire Quickbook support, they can
> make a tool to convert from ledger. There's no good reason for a
> non-profit, libre software organization to use and depend on
> proprietary software. Did nobody learn a lesson from BitKeeper?

Have you checked that the people hosting Gentoo's infrastructure don't
use any proprietary software anywhere inside their building either? Most
cleaning companies use a closed source staff scheduling program. It
would be a terrible violation of the social contract if Gentoo depended
upon that.

-- 
Ciaran McCreesh

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[Ian Stakenvicius]

> On Dec 5, 2016, at 5:01 AM, Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> 
> On Sun, 4 Dec 2016 18:47:48 -0800
> Daniel Campbell <zlg@gentoo.org> wrote:
>> Compliance with what? If others desire Quickbook support, they can
>> make a tool to convert from ledger. There's no good reason for a
>> non-profit, libre software organization to use and depend on
>> proprietary software. Did nobody learn a lesson from BitKeeper?
> 
> Have you checked that the people hosting Gentoo's infrastructure don't
> use any proprietary software anywhere inside their building either? Most
> cleaning companies use a closed source staff scheduling program. It
> would be a terrible violation of the social contract if Gentoo depended
> upon that.
> 

Fortunately Gentoo does not require any cleanliness standards be met at all.  Otherwise I couldn't be a dev. :P

That is a fairly important distinction here, I think.  If we start using or relying on Quickbooks then we are locking into that proprietary platform--getting all the historical financials out of Quickbooks and into another tool is intentionally hard (I assume--I know it's near impossible with Sage).  We are not requiring or accountant to use open tools though, if they want to import our data into Quickbooks to do their work, that's up to them, as long as we still have the result we need from them.

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[james]

On 12/04/2016 06:49 PM, Robin H. Johnson wrote:
> On Sun, Dec 04, 2016 at 11:07:59PM +0000, M. J. Everitt wrote:
>> I gather both Quickbooks and Sage have a more modular approach to
>> "proper" accounting software applicable to small and large businesses. I
>> know my mother used Quickbooks in the past with good success and the
>> support of her accountant, but Sage is known to be equally accessible. I
>> would imagine there is an appropriate version for not-for-profit or
>> charities, perhaps you can seek advice with the person(s) already
>> contacted for accounting/finance purposes?!
> Our CPA (Yes, we do have one) only recommends QuickBooks, but has used a
> variety of other proprietary systems (none of which he recommends at
> all!).
>
> The catch is that either Quickbooks or Sage would be a violation of the
> social contract's libre-licence dependence clause.
>
> Ledger HAS filled most of our needs thus far, but lacks in reporting and
> some automation:
> - I'd love to automatically generate lots of depreciation
>   entries, but can't yet.
> - Something to anonymize private information in some entries, so that
>   the actual Ledgers can be published for transparency.
>

All of that is routine and easy with GNUcash....


hth,
James

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[M. J. Everitt]

[-- Attachment #1: Type: text/plain, Size: 1956 bytes --]

On 05/12/16 03:06, james wrote:
> On 12/04/2016 06:49 PM, Robin H. Johnson wrote:
>> On Sun, Dec 04, 2016 at 11:07:59PM +0000, M. J. Everitt wrote:
>>> I gather both Quickbooks and Sage have a more modular approach to
>>> "proper" accounting software applicable to small and large
>>> businesses. I
>>> know my mother used Quickbooks in the past with good success and the
>>> support of her accountant, but Sage is known to be equally
>>> accessible. I
>>> would imagine there is an appropriate version for not-for-profit or
>>> charities, perhaps you can seek advice with the person(s) already
>>> contacted for accounting/finance purposes?!
>> Our CPA (Yes, we do have one) only recommends QuickBooks, but has used a
>> variety of other proprietary systems (none of which he recommends at
>> all!).
>>
>> The catch is that either Quickbooks or Sage would be a violation of the
>> social contract's libre-licence dependence clause.
>>
>> Ledger HAS filled most of our needs thus far, but lacks in reporting and
>> some automation:
>> - I'd love to automatically generate lots of depreciation
>>   entries, but can't yet.
>> - Something to anonymize private information in some entries, so that
>>   the actual Ledgers can be published for transparency.
>>
>
> All of that is routine and easy with GNUcash....
>
>
> hth,
> James
>
>
>
Grabbing the bull by the horns here, any willing/able volunteers to aid
robbat2 getting ledger ported to gnucash and up-to-speed maybe? I can't
really volunteer as I'm not good with finance esp. not US and have one
too many pans in the fire right now...! :)

For zlg's benefit .. I wasn't advocating re-writing the social contract
(yet) just questioning whether that may be an unhelpful constraint in
quite an important process, but I sit corrected in that there are libre
solutions to this issue in use in similar environments .. so we just
need to transition ..

2c50 !


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[james]

On 12/04/2016 10:10 PM, M. J. Everitt wrote:
> On 05/12/16 03:06, james wrote:
>> On 12/04/2016 06:49 PM, Robin H. Johnson wrote:
>>> On Sun, Dec 04, 2016 at 11:07:59PM +0000, M. J. Everitt wrote:
>>>> I gather both Quickbooks and Sage have a more modular approach to
>>>> "proper" accounting software applicable to small and large
>>>> businesses. I
>>>> know my mother used Quickbooks in the past with good success and the
>>>> support of her accountant, but Sage is known to be equally
>>>> accessible. I
>>>> would imagine there is an appropriate version for not-for-profit or
>>>> charities, perhaps you can seek advice with the person(s) already
>>>> contacted for accounting/finance purposes?!
>>> Our CPA (Yes, we do have one) only recommends QuickBooks, but has used a
>>> variety of other proprietary systems (none of which he recommends at
>>> all!).
>>>
>>> The catch is that either Quickbooks or Sage would be a violation of the
>>> social contract's libre-licence dependence clause.
>>>
>>> Ledger HAS filled most of our needs thus far, but lacks in reporting and
>>> some automation:
>>> - I'd love to automatically generate lots of depreciation
>>>   entries, but can't yet.
>>> - Something to anonymize private information in some entries, so that
>>>   the actual Ledgers can be published for transparency.
>>>
>>
>> All of that is routine and easy with GNUcash....
>>
>>
>> hth,
>> James
>>
>>
>>
> Grabbing the bull by the horns here, any willing/able volunteers to aid
> robbat2 getting ledger ported to gnucash and up-to-speed maybe? I can't
> really volunteer as I'm not good with finance esp. not US and have one
> too many pans in the fire right now...! :)



Well, the key skill that is needed is someone who has the vision of what 
the 'chart of accounts' needs to look like for a 501(c) gentoo 
organization. The gnucash-user list is full of helpful folks that will
help with migration, including numerous scripts that automate conversion
from other accounting systems to gnucash. Really the next step would be 
for Robatt2 figure out and list what he wants. I'm already subscribe to
gnucash-user, so if a few other folks did likewise it would get the ball 
rolling. One could even state the type of 501(c) and ask for a suggested 
list of chart of accounts for that type of business organization, on the 
gnucash-user list. I have already sent robatt2 a coulple of contacts 
that currently run 501(c) organizations on gnucash.


Define what you have and what you want to do, then ask for help on 
gnucash-user list for ideas and guidance. I do not believe that any 
custom programming is needed, but I''m not familiar with the needs of 
501(c) in general, nor any of the gentoo specific needs.


hth,
James


> For zlg's benefit .. I wasn't advocating re-writing the social contract
> (yet) just questioning whether that may be an unhelpful constraint in
> quite an important process, but I sit corrected in that there are libre
> solutions to this issue in use in similar environments .. so we just
> need to transition ..
>
> 2c50 !

I would never use FOSS accounting software because it is FOSS. I use 
gnucash, after working with accountants on dozens of PC based packages
and it is simple the best, flexible, free support and many have custom 
scripts that that share, freely, that makes gnucash my goto package for 
accounting. gnucash does no 'lock in your data' and that is a big deal 
for most organizations. Gnucash makes it easy to do what you want, with 
a robust, double entry system that has countless cool features.


ymmv,
James

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[james]

On 12/04/2016 05:55 PM, Robin H. Johnson wrote:
> (OT accounting systems)
>
> On Sun, Dec 04, 2016 at 01:10:16PM -0500, james wrote:
>> GNUcash is superior to Quickbooks, as it is a 'double entry' accounting
>> system. Last time I check Quickbooks was not 'double entry' and that is
>> a big deal in accounting.
> QuickBooks is double-entry, and has been for a very long time; It did
> used to obscure the fact before, to make accounting 'easier' for
> non-accounting people to understand.
>
> For the Foundation, I'm presently using Ledger-CLI [1], but this is a
> hurdle for any third-party financial auditing (we should be prepared at
> all times for a real financial audit), because they want data in
> quickbooks format.

GNUcash can import/export  any number of common/proprietary formats
including quickbooks. Quickbooks serves as the largest base of 
frustrated users that migrate (routinely) to gnucash, particularly
for custom and unique and open needs. Gnucash, found in the protage tree
has these flags::     chipcard debug doc gnome-keyring hbci mysql ofx 
postgres python quotes sqlite

So it looks like you have a choice of sql mechanisms to aid your 
customization needs. Combine that with the Android terminal feature and 
that means that lots of fiduciary oriented folks at gentoo could file 
reports and make/parse entry data, so you have a responsible team of 
folks in the 'accountability-matrix' at the gentoo foundation.


> If there is a good GNUCash support for non-profit accounting (which does
> differ from small-business accounting, see [2]), and matching
> documentation for it, I'm VERY interested to know about it.


GNUcash is very open and I've read about all sorts of custom reports and 
modules for a plethora of varied needs (gnucash-user). My company needs 
are very modest, so I have not ventured into customizing gnucash. When I 
take my annual reports to my tax accountant (mid sized Accounting firm) 
they are always impressed with the quality of the reports and the fact I 
can at anytime print a complete ledger, or specify to/from dates, with 
detailed annotations of all events/transactions/anomalies.

In 27 year of having fiduciary responsibilities at a variety of 
organizations, none of my teams/companies have been audited by the IRS. 
Squeaky, tight-assed accounting and knowing your business, is reflected 
in the team and the documents you send to the IRS; it's just that 
simple, regardless of organizational structure. A 'tight-ass' is a 
tight-ass, reflected in your documents and 2 minutes of browsing by a 
seasoned CPA/auditor and they know more about you than you do. In 
(tax)accounting, there are only A+ and F participants, imho.


Here are a few links you can ponder, before joining the mailing list and
formulating specific questions as to what you want(need) in an 
accounting package for a non-profit.

http://wiki.gnucash.org/wiki/Custom_Reports#Get_to_know_Scheme

http://wiki.gnucash.org/wiki/Custom_Reports_Using_Eguile

In your second reference, I do not see any thing that gnucash is not 
able to do; mostly with a judicious naming and organization of your 
'chart of accounts'. In fact the only report that does not have a 
functional equivalent is the 'Statement of Functional Expenses'. But 
that should not be too difficult to create; but maybe not to your standards?


Standard reports required for example "Net assets" have a gnucash 
equivalent, which could be further customized to your liking.

Things like "Nonprofit expenses are reported by these functions::
Program, management and general, and fundraising", would be handled
structurally as your 'chart of accounts' where you define the structure. 
Perhaps you need an accountant to help you define your 'chart of accounts'?

Surely Gentoo has as specific CPA advising the organization?
If you are in  So. Calif, I know a cranky old vet, that worked for the 
IRS and is brilliant. He's a bit of an 'old bastard', but he has a soft 
touch.  He's not too computer literate, but he know the IRS inside and 
out. He might help you out in exchange for putting gentoo on an old 
lappy? In fact, he can show you how to read 'Title 26' of the IRS code 
for goodies not found in those little pamphlets that are publish by the 
IRS, should you need a 'deep dive', or ever experience an audit.....
But that is no substitute for an extended relationship with a practicing
CPA over the long term. The aforementioned expert can get you where you 
need to be, for little in the way of compensation. If you take a mess to 
a CPA, it's going to cost theGentoo Foundation (GF) a bundle to sort 
out. If you clean it up, first, and walk in with pristine reports and 
documentation and the 'old bastard' the CPA will not be able to rape you 
(financially). Caveat emptor.


Link (3) looks like a fine organization, but, I'd make sure there is 
some real experience on that team, as the pres_elect is prolly going
to smack lots of tech endeavers, centric to Calif. around a bit once in 
office; just like Obama did to the christian charities (remember?). In 
fact the republican congress, defunded the IRS a few years to stop the 
attack by the Obama Administration of 501(c) churches, if you recall 
your tax politics. What the first thing Trump is doing? Diddling the tax 
codes, as do most presidents. In fact, I'd be a bit surprised if he did 
not 'clean up the 501(c) space a bit, just to reclassify many 501(c) 
into a 'hobby or a private club', thus bringing in a lots of cash into 
the IRS.


Recall, that Lyndon Johnson, put the gag order on the churches in the 
1960s (if you speak out politically, you loose your tax-exempt status) 
and all executive branches of government have since 'tighten the noose' 
since Lydon Johnson on those 501(c) designated churches. Heartless 
bastards, including Ronald Reagan, imho. They loose their tax status, 
they pay local real-estate property taxes, which would not upset many 
local taxing authorities.  In accounting, you need to 'read the tea 
leaves' of what the new administration intends to do.  Trump, like no 
other executive before him, directly implicated Lyndon Johnson, 
repeatedly, for this abhorrent precedence, emanating for the 1960s. So 
do not think for a second, that the IRS will go light on any 501(c).
What you see as noble, will be judged a 'tax dodge' if your records are 
not in order; count on it. A squeezing of the 501(c) space, is long past 
due and always brings in lots of cash into the IRS.


> Why Ledger? The Software Freedom Conservancy started a project aimed at
> Non-Profit accounting [3], wrapped around Ledger, which covers far more
> of the non-profit nuances than GNUCash does.


That's just stupid, to pine your defense against the IRS around a piece 
of software. That foundation better have 'real expertise' as in former 
IRS agents as part of the team, or CPA-Lawyers, that know haw to read 
and argue Title-26 as a basis of support. The numbers do not make any 
difference without fundamental (tax) accounting expertise. In fact they 
should support a variety of soft wares that mapp into their knowledge 
base of the laws formed around title-26, or it's a fools errand. 
Granted I do not know that organization, and they are probably good 
people. But when it comes to a scrap with the IRS, fundamental knowledge 
of tax codes, is all that counts. Defensibility of intent, when correct 
establish means errors are just that, errors that need correcting. An 
incorrect collection of timely documentation and the organization can 
easily get smacked around (huge fines) and even reclassified as a hobby, 
or a good ole boys club.


However, I'm no expert on this, but gnucash is very flexible in how you 
create your chart of accounts, but I'm not certain that plus custom
reports and a robust sql interface will allow you to customize gnucash 
the way you have in mind, or the way your mentors have in mind.



> They included enough documentation in how to specifically configure
> Ledger for non-profit usage, so it was easy to get going since I already
> used Ledger for my personal accounting.

GNUcash is a lot like Gentoo. You can pretty much do anything with it, 
but you have to look at the project, as a labor-of-love. It's FOSS, so
all the code is there, C and scheme, mostly.


> Ledger being plain-text based does work very well with version control,
> even for multiple parties (I enlisted help to convert old bank
> statements).

Gnucash has many robust ways to get data from old/proprietary systems 
upload to gnucash or a sql database.

> [1] http://www.ledger-cli.org/
> [2] http://www.accountingcoach.com/nonprofit-accounting/explanation/1
> [3] https://sfconservancy.org/npoacct/


Link (3) sounds great. What is their position on GNUcash? I'd think 
they'd support many different FOSS solutions? Your looking for expertise
and it needs to come from someone you can sit down with and work on 
things, including all those 'strategic vision' types of records. In my 
experiences, when you change accounting systems, you keep the first one 
up and running (pristine) and bring up and customized the new accounting 
system, side-by-side for at least one full (cycle) year,
if not two.

It sounds to me as though the old 'trust but verify' saying means you GF 
needs to have an independent audit on where it is at. In times of 
uncertainty, an Audit is  fiduciary responsibility that should not be 
ignored.

Please let me know (privately) if I can help further. Only fools do not 
prepare for an IRS audit. If you prepare ahead of time, they will not 
bother the GF. If you don't................. I actually hate accounting
and I *fear the irs* but, I've seen the destruction they leave, when 
they choose to smack down tax-dodges...... and 501()c) is no exception. 
  ymmv.


hth,
James

Re: [gentoo-dev] (OT) Accounting systems: Ledger-CLI vs GNUcash

[james]

On 12/04/2016 05:55 PM, Robin H. Johnson wrote:
> (OT accounting systems)

> If there is a good GNUCash support for non-profit accounting (which does
> differ from small-business accounting, see [2]), and matching
> documentation for it, I'm VERY interested to know about it.

Robin,

I posted on on the gnucash list and got some responses. You should join 
that list and get your detailed questions answered. Gnucash has a 
wonderful collection of expertise on that list and they appear to be
many 'non-profits' using gnucash and they are quiet helpful::

I posted this::


On Sun, Dec 4, 2016 at 6:18 PM james <garftd@verizon.net> wrote:
     Hello gnucash users.
     I use gnucash for my small business, for years and I'm quite happy 
with     it. Recently, I was ask if Gnucash has as good of support for 
501(c)3     non-profits as does ledger (www.ledger-cli.org)?
     Any and all comments are warmly received.

     James


And the private response was::

"At its heart, anything you can do with a pen-and-paper system Double 
Entry Accounting system, you can also do with GnuCash. This includes 
keeping books for a 501(c)(3). Several of us do so.

There are a few things you might want to customize: the "Profit/Loss" 
report is misnamed for a non-profit organization, for instance, and the 
standard business chart of accounts does not match the categories that 
the IRS wants things to be in for the annual tax filing. But those are 
all easy to change."


So just join gnucash-user and get a solution you are happy with.

hth,
James

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 4965 bytes --]

On 12/04/2016 10:10 AM, james wrote:
> On 12/04/2016 02:22 AM, Robin H. Johnson wrote:
>> On Sat, Dec 03, 2016 at 06:30:29PM -0500, William L. Thomson Jr. wrote:
>>>> ----------------
>>>> Net Total: $50,924.19
>>>> ----------------
>>> So from 09-16 avg of ~$4.6k per year over 11 years.
>> 10 years of participation, 9 of which we got paid for. So ~$5.7k/year.
>> If we got paid for 2013: ~$5.4k/year over 10 years.
>>
>>> With that really being earned by people doing GSoC. Not the same as if
>>> Google donated a lump sum of money to further development per say the
>>> Councils plans. Only 1 hardware donation.
>> That's the payment to the organization for mentoring and managing the
>> students, separate from what the students doing GSoC earned.
>>
>> If the student's work was of use to Gentoo, then it's ALSO $5000-$5500
>> per student that we've had in man-hours. I do use that disclaimer,
>> because I know the integration rate for Gentoo students much lower than
>> it should be.
>>
>> 2006: 10 students
>> 2007: 8 students
>> 2008: 5 students
>> 2009: 6 students
>> 2010: 16 students
>> 2011: 14 students
>> 2012: 8 students
>> 2013: 6 students
>> 2014: 3 students
>> 2016: 5 students
>>
>> Total: 81 students.
>> Assuming $5k/student: $405,000 in student payments, over 11 years.
>>
>> I don't know how many students we've failed: I do know it's been at
>> least one (I failed them. Their original mentor had medical issues, I
>> took over, and they provided a mocked video of their work and no code by
>> midterm).
>>
>>> I believe past sponsors such as GNi incurred costs in the ~$5k range
>>> monthly.
>>> I would assume some hosting sponsors to be averaging a few thousand
>>> at minimum
>>> per year.
>> The cost to GNi was much closer to $1k/month, mostly in potential lost
>> revenue if the hardware COULD be used for income (it was already a sunk
>> cost, and didn't have other users). For our present major hosting
>> sponsors, I believe we're more in line with $250-$400/month, but again
>> mostly older hardware that isn't of much other salable use.
>>
>>> Just as an example. FreeBSD is seeking $1.25 Million in a fundraiser
>>> with
>>> $882k thus far.
>>> https://www.freebsdfoundation.org/
>> $1.25M is their annual fund-raising target for this year and last. Not a
>> specific fund-raiser, but their annual target.
>> For 2016 Q1-Q3, on the $1.25M, they report $293k in contributions.
>> For 2015, on a $1.25M target, they reported $657k in contributions.
>> For 2014, on a $1M target, they reported $2.4M in contributions.
>>
>>> They seem to average in the hundreds of thousands every year in
>>> contributions
>>> https://www.freebsdfoundation.org/about/financials/
>> They're also got a good few years on us (as do Apache).
>>
>>> Always looked at FreeBSD when I was a Gentoo Trustee. Great
>>> foundation! Passed
>>> the 5 year probation period with IRS, and other stuff.
>> The Apache Foundation was very beneficial to look at I found, because
>> they kept superb public records, but also were not hampered by some of
>> our restrictions about depending on non-open software (they & the perl
>> foundation BOTH use QuickBooks on Windows for their accounting).
> 
> 
> GNUcash is superior to Quickbooks, as it is a 'double entry' accounting
> system. Last time I check Quickbooks was not 'double entry' and that is
> a big deal in accounting.  There is a module that allows entries via
> Android now with GNUcash, but is not an official part of GNUcash.org. I
> use GNUcash with my company, but not the Android smartphone module.....
> 
> 
> http://gnucash.org
> 
> http://www.techrepublic.com/article/gnucash-a-powerful-mobile-financial-tool-for-android/
> 
> 
> 
> Serious inquires could be directed to 'gnucash-user@gnucash.org' as this
> accounting software is robust, under active development and even the
> devs 'chime in' on  routine basis.  All in all, gnucash is an
> outstanding piece of FOSS software; much better than Quickbooks as many
> on the discussion lists attest to on a routine basis. It is in portage
> and it runs on windows and other platforms.
> 
> 
> hth,
> James
> 
> 
>> https://www.apache.org/foundation/records/
>>
>> I draw your attention to their last 990 filing:
>> https://www.apache.org/foundation/records/990-2014.pdf
>> - $1.2M in annual income
>> - $858k spend on infrastructure,
>>   of which >$400k was marked directly as IT spending.
>> - $1.8M in net assets
>>
> 
> 
iirc, we're using Ledger (http://ledger-cli.org), which is also
double-entry accounting. It uses a text file for its information, and
has a ton of reporting features that make it trivial to produce reports.
I use it to manage my personal finances, as well.

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 2773 bytes --]

On 12/03/2016 03:01 PM, Robin H. Johnson wrote:
> On Sat, Dec 03, 2016 at 04:49:19PM -0500, William L. Thomson Jr. wrote:
>> No matter what terms you use, Google uses Gentoo to build products it makes 
>> money off. In that sense I think it could give back allot. If not in 
>> donations, resources (tinderbox), etc.
> Here's the financial parts of what Google has given us, that I can
> quantify. It DOESN'T include anything of the 20% time that might have
> been used in Gentoo's favour [some past Google-employed developers have
> specifically said they were spending Friday afternoons doing Gentoo
> dev], or paid Gentoo stuff that overlapped their actual job needs
> (ChromeOS-related).
> 
> Non-GSOC:
> ---------
> 2011: Google donated brand-new Dell servers, that with their volume
> discount, had an invoice price of $4331.60, and a Dell list price over
> $6k. Those servers are hosted at OSUOSL, and are still in active use
> (dipper, blackcap). 
> Net Subtotal: $4,331.60
> 
> GSOC:
> -----
> Gross Payments
> (including reimbursement for mentor summit travel expenses)
> 2009:  $5,151.59
> 2006:  $7,000.00
> 2007:  $4,500.00
> 2008:  $3,000.00
> 2010: $11,001.25
> 2011:  $9,891.77
> 2012:  $7,000.00
> 2013:      $0.00 [1]
> 2014:  $4,200.00
> 2015:      $0.00 - did not participate
> 2016:  $4,700.00 [2] 
> Gross Subtotal: $56,444.61
> Less reimbursed travel expenses: $9,852.02
> Net Subtotal: $46,592.59
> 
> ----------------
> Net Total: $50,924.19
> ----------------
> 
> [1] 
> - Details in bug 488142. Not locked, no personal information in it
>   (other year GSOC bugs are locked due to containing mentor personal
>   information).
> - No record could be found for a 2013 invoice, in our bank records OR
>   Google's invoice archive [MANY thanks to Antarus and the Google
>   accounting department who did pull all historical invoices submitted
>   by Gentoo].
> - presumed we did not submit before the deadline and thus forfeited the
>   payment & reimbursement.
> - Net Amount if we had filed: $2591.55
> -- 6 students * $500/student: $3000
> -- Less $2608.45 in actual travel expenses (max $2200 reimbursement)
> - Net for non-filing: $-2608.45
> - NOT included in reimbursement subtotal above.
> 
> [2] Invoice submitted 2012/11/29, payment NET30 period ends 2016/12/29
> 
I just wanted to point this e-mail out and thank you for the effort
spent to share information like this. This is a great step, and once we
get the books in order, sharing this information using automated means
could get us part of the way to 501(c)3 status.

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1353 bytes --]

On Saturday, December 3, 2016 11:42:14 PM EST Daniel Campbell wrote:
>
> I just wanted to point this e-mail out and thank you for the effort
> spent to share information like this. This is a great step, and once we
> get the books in order, sharing this information using automated means
> could get us part of the way to 501(c)3 status.

I hope that is still possible, and I am not sure if it was even in 07-08. I am 
not tax expert, not a CPA or anything close. Likely need to retain and speak 
to one. May have to refile and start over not sure.

There is basically a 5 year "probation qualification" period that starts with 
the first year of the NPO applying for 501c status.
https://www.irs.gov/publications/p557/ch03.html#en_US_201602_publink1000246837

Info on the tests and qualification requirements
https://www.irs.gov/publications/p557/ch03.html#en_US_201602_publink1000200142

I have no clue what happens if you do not meet the requirements in the 5 
years. Not sure if you can re-apply or what. Thus need to hire and speak to 
someone, if not have them help to get things in order.

There are presently larger issues. I will let others speak on that, as I am 
not sure how much has been released on such. Though hiring a CPA or other may 
be able to help with that, and can determine the direction of the rest.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 469 bytes --]

>>>>> On Sun, 04 Dec 2016, William L Thomson wrote:

> I hope that is still possible, and I am not sure if it was even in
> 07-08. I am not tax expert, not a CPA or anything close. Likely need
> to retain and speak to one. May have to refile and start over not
> sure.

> [...]

Can you please move this discussion elsewhere, e.g. to the gentoo-nfp
or the gentoo-project list? It certainly is of non-technical nature
and therefore doesn't belong in gentoo-dev.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 2898 bytes --]

On Sat, Dec 3, 2016 at 6:00 AM, William L. Thomson Jr. <wlt-ml@o-sinc.com>
wrote:

> OT, who runs Gentoo
>
> On Saturday, December 3, 2016 12:21:55 AM EST Daniel Campbell wrote:
> >
> > Unless something's changed in the past year or two, iirc Sony uses
> > Gentoo as part of the backend of Gaikai, Google's used it for the base
> > of ChromeOS...
>
> Also is the base of CoreOS, widely popular. Google uses it in more than
> ChromOS it was used in the OnHub router. I suspect Google has wider uses.
>
> Google has hired a few core developers as has Gaikai. Both seem to be good,
> though not sure Google is giving back as much given their financial
> benefit.
> Gaikai isn't selling an OS, but Google is based on Gentoo...


> > I can't speak for other 'big names', but Gentoo's not
> > quite as niche as the small, active userbase has most of us believing.
>
> FYI Network Solutions, which I believe at a time had a root DNS server
> running
> Gentoo. Due to their parent company Web.com is, or has by now, moving from
> Gentoo to RHEL. I was told they do not want to be in the operating system
> development business.
>
> Meetup.com runs Gentoo, not sure how widely but it is in use. Likely more
> out
> there, but I see far more RHEL and CentOS. I do not even discuss Gentoo,
> as I
> have to many say it can't be used in production, etc. Things I disagree
> with.


> > There's also our downstream neighbors: Funtoo, Pentoo, Sabayon,
> > Calculate, Exherbo, etc
>
> Two of those are more of a splinter of the Gentoo community, Exherbo and
> Funtoo. IMHO those communities should be reunited into Gentoo. Allot of
> talent
> got driven way that is still out there working on other stuff.
>
> > As for communities, lots of places from 4chan to lainchan, various mesh
> > network users, security-conscious communities, OCD support groups
> > (kidding), etc.
>
> My concern is lack of any mention in the news or tech articles. When was
> the
> last time Gentoo was making headlines and not for drama related stuff.
>

I don't think making headlines is a goal for the project. Maybe you think
it should be; but I don't see leadership (on the Council or the Trustees)
pushing in that direction. So while you can have the opinion that the
distro is dead (or going downhill or whatnot) many people do not think this.

-A


>
> Gentoo News articles
> https://www.google.com/search?hl=en&gl=us&tbm=nws&authuser=
> 0&q=gentoo+linux
>
> Arch News articles for example, other distros Ubuntu and Debian have allot
> https://www.google.com/search?hl=en&gl=us&tbm=nws&authuser=0&q=arch+linux
>
> Not my words but like say Infoworld articles, not that they are any
> authority
>
> http://www.infoworld.com/article/3109830/linux/why-did-
> gentoo-linux-fade-into-obscurity.html
>
> http://www.infoworld.com/article/3137969/linux/arch-
> linux-the-last-refuge-for-purists.html
>
>
> --
> William L. Thomson Jr.
>

[-- Attachment #2: Type: text/html, Size: 4514 bytes --]

Re: [gentoo-dev] OT Who runs Gentoo was -> RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 440 bytes --]

On Saturday, December 3, 2016 2:40:43 PM EST Alec Warner wrote:
>
> I don't think making headlines is a goal for the project. 

Making headlines and having attraction is usually something that matters in 
technology. Lots of obscure tech out there, no one writes about or cares 
about. Which leads to less usage, less contributions, less interest, etc.

Its not a goal, its just a sign of relevance.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 4255 bytes --]

On Monday, November 28, 2016 02:39:48 PM William L. Thomson Jr. wrote:
> On Monday, November 28, 2016 10:42:54 AM EST Alec Warner wrote:
> > Generally speaking as a fellow who maintained thousands of systems (many
> > of
> > which ran various operating systems.)
> > 
> > You cannot rely on all OS vendors to synchronize uid / gid. You cannot
> > even
> > rely on some single vendors to synchronize uid / gids between releases of
> > their own products.
> 
> I believe the main reason such is the case is a lack of any such list or
> database for others to adhere to. Once again an area Gentoo could be
> leading. Had Gentoo done this years ago others might have adopted.
> 
> IMHO it is something that should be  a part of LSB. If not POSIX in general.
> One cannot really change the past or current state of things. But can make
> the future better.

Highly detailed lists like that--used as a broad standard--are a bad idea. 
They represent a single synchronization point that everyone must adhere to. 

That means that every prospective adjustment to the list requires active 
maintenance. That means that for every new daemon someone writes, they have to 
go through an admissions process. For every contentious fork of a project, you 
risk conflict over who the designated contact for the assignment should be.

It adds a large bureaucratic load on everyone. Every itch some developer 
thinks about scratching has to be weighed against engaging with some process-
laden entity. Maybe they'll participate, but they likely won't.

Have you watched the IANA ports assignment registry over the years? Consider 
how many services and tools you've seen that *don't* respect it.

And what is the list managing? A limited namespace, currently only 32 bits, 
but with tools like, say, Samba and sssd reserving large chunks for stable UID 
and GID mapping. One could argue that a stable list could obviate the need for 
some of that mapping, but you've got decades-old existing networks that aren't 
going anywhere, and you'll still need to interface with systems run by people 
who will deliberately run counter to such lists as a security layer, just as 
you interface with systems that run SSH or HTTP on nonstandard ports.

You'll still run into all of these issues and more if you try generalize the 
list to region allocation:

Say you try to assign regions for system daemons vs users, and you're on a 
host that interacts with two other hosts without full mutual trust. You're 
serving up a shared filesystem, and all three involved hosts each have a system 
daemon user and a system normal user with an object on that shared filesystem.

Presented with a directory listing showing the UIDs and GIDs for each object, 
how do you distinguish between the system user from each host? The two hosts 
shouldn't have access to each others' files, even if they use the same UID 
locally, because the two hosts don't trust each other.

That considered, how, then, how do you identify between another host's system 
user and its normal user, inasmuch as you can't let them collide with IDs on 
your own system, but are trying to ensure that their IDs get mapped into the 
correct local range?

That considered, what do you do when the Big List Maintainers add another 
region? How do you cope with another host that uses a newer version of that 
list? An older version? And now that you've upgraded, and the new version of 
the list says you should have mapped something somewhere else, what do you do 
with it? Do you even have enough information to know that an ID you mapped 
last year should have been in that other category?

And while we're talking about allocating ranges, we can start talking about 
limited address space. 32 bits seems like a lot of individual identities, but 
when you're carving it up into multiple masses of identities, you'll find it 
gets very small, very quickly. That's why IPv6 went with 128 bits instead of a 
64 bit address space.

All of this is why we use identity management tools like LDAP in the first 
place. Heck, it's why we have passwd and group files for mapping names to ids 
and didn't simply hardcode system IDs decades ago.

-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 2898 bytes --]

On Tuesday, November 29, 2016 10:40:20 AM EST Michael Mol wrote:
>
> 
> Highly detailed lists like that--used as a broad standard--are a bad idea.
> They represent a single synchronization point that everyone must adhere to.

That is a statement based on opinion. You say it is a bad idea. I say it is 
necessary and needed. Otherwise wrt to Gentoo ebuilds can stomp on each other. 
Using same GID or UID in more than one ebuild causing problems. There has to 
be something know so others do not use ones others are already.

> That means that every prospective adjustment to the list requires active
> maintenance. That means that for every new daemon someone writes, they have
> to go through an admissions process. For every contentious fork of a
> project, you risk conflict over who the designated contact for the
> assignment should be.

If they package such in Gentoo someone is making a call as to what UID and GID 
should be used. If you think about it from packaging said new daemon in 
Gentoo, it is a MUST.

If it does not exist, should it be entirely random from the packager 
perspective? What if they use a GID/UID specific to them and not others.

There has to be some standard some consistency in Gentoo.

> It adds a large bureaucratic load on everyone. Every itch some developer
> thinks about scratching has to be weighed against engaging with some
> process- laden entity. Maybe they'll participate, but they likely won't.

Gentoo shines at bureaucratic load. That may be one of the only things Gentoo 
is really good at, needless bureaucratic loads that just slow things down and 
fracture the community, exherbo, funtoo, and likely others...

This is not needless bureaucracy , this is necessary.

> Have you watched the IANA ports assignment registry over the years? Consider
> how many services and tools you've seen that *don't* respect it.

Yes, how often to ports < 1024 change? Hardly ever.... Proving the exact point 
why this is needed. People can change them themselves but 99% of the time its 
to some other port > 1024.

Why is there IANA port assignment registry in the first place? Likely for a 
similar reason.

> All of this is why we use identity management tools like LDAP in the first
> place. Heck, it's why we have passwd and group files for mapping names to
> ids and didn't simply hardcode system IDs decades ago.

LDAP typical manages user accounts not system. If the LDAP server is not 
reachable you would make a system completely nonfunctional if it relied on 
LDAP for system accounts.

Also needed from a file sharing stand point of view if sharing parts of a 
system across others. You need consistent GID/UID mappings or things like NFS 
will have lots of problems.

Package a few things in Gentoo that need a UID and/or GID and you will start 
to understand the problem from a operating system packager perspective.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Alan McKinnon]

On 29/11/2016 23:49, William L. Thomson Jr. wrote:
> On Tuesday, November 29, 2016 10:40:20 AM EST Michael Mol wrote:
>>
>>
>> Highly detailed lists like that--used as a broad standard--are a bad idea.
>> They represent a single synchronization point that everyone must adhere to.
> 
> That is a statement based on opinion. You say it is a bad idea. I say it is 
> necessary and needed. Otherwise wrt to Gentoo ebuilds can stomp on each other. 
> Using same GID or UID in more than one ebuild causing problems. There has to 
> be something know so others do not use ones others are already.

Why would you end up with duplicated UIDs and GIDs? The only real ways
that can happen is
- ebuild "edits" passwd and group directly using echo/sed and the like.
- ebuild runs useradd|groupadd specifying the uid/gid as arguments

Both of which are silly. Just use useradd/groupadd without uid/gid
arguments. The utility will make sure the uid/gids are non-duplicate,
and ensure they are <1000 or whatever for system accounts

> 
>> That means that every prospective adjustment to the list requires active
>> maintenance. That means that for every new daemon someone writes, they have
>> to go through an admissions process. For every contentious fork of a
>> project, you risk conflict over who the designated contact for the
>> assignment should be.
> 
> If they package such in Gentoo someone is making a call as to what UID and GID 
> should be used. If you think about it from packaging said new daemon in 
> Gentoo, it is a MUST.

How do you intend to MAKE devs follow it? More eternal bike-shedding?

> If it does not exist, should it be entirely random from the packager 
> perspective? What if they use a GID/UID specific to them and not others.
> 
> There has to be some standard some consistency in Gentoo.

Who cares what the uid/gid is? There's a range of about 950 to chose
from. The way to ensure a filesystem object has the correct owner and
group is by using chown/chgrp.

Except for a few cases out on left field (like nfs shares - a problem
that nfs must fix) you don't really care what the uid/gid is, as long as
it's not duplicated. The thing you care about is the NAME

> 
>> It adds a large bureaucratic load on everyone. Every itch some developer
>> thinks about scratching has to be weighed against engaging with some
>> process- laden entity. Maybe they'll participate, but they likely won't.
> 
> Gentoo shines at bureaucratic load. That may be one of the only things Gentoo 
> is really good at, needless bureaucratic loads that just slow things down and 
> fracture the community, exherbo, funtoo, and likely others...
> 
> This is not needless bureaucracy , this is necessary.

This is a joke right?

>> Have you watched the IANA ports assignment registry over the years? Consider
>> how many services and tools you've seen that *don't* respect it.
> 
> Yes, how often to ports < 1024 change? Hardly ever.... Proving the exact point 
> why this is needed. People can change them themselves but 99% of the time its 
> to some other port > 1024.
> 
> Why is there IANA port assignment registry in the first place? Likely for a 
> similar reason.

It's so that things like browsers, email tools and the like can drop
:<port> for the most part and be reasonably sure stuffs will still work.

Of the 65535 +-1 possible port numbers, only the first 1024 are truly
important, and of those less than about a quarter are in common use
(wild guess).

The next 10,000 or so are not standards by any means, just a list of
stuff that happens to have been seen in the wild. Apps can and do pick
any old port they feel like - witness the several things that will use
5000 out the box. Is this a problem? Not really, as very very few
machines out there will install two apps both trying to use port 5000 by
default.

The top 45,000 - well that's a free-for-all. Mostly only used as source
ports used by apps trying to contact other apps, and not by listeners.

When looked at IANA port assignments in this light, it really does seem
to be a case of the minimum necessary bureaucracy to make things mostly
nicely most of the time, and not at all a case of bureaucracy to
standardize things as you seem to be arguing.

> 
>> All of this is why we use identity management tools like LDAP in the first
>> place. Heck, it's why we have passwd and group files for mapping names to
>> ids and didn't simply hardcode system IDs decades ago.
> 
> LDAP typical manages user accounts not system. If the LDAP server is not 
> reachable you would make a system completely nonfunctional if it relied on 
> LDAP for system accounts.
> 
> Also needed from a file sharing stand point of view if sharing parts of a 
> system across others. You need consistent GID/UID mappings or things like NFS 
> will have lots of problems.

NFS is an edge case. Maybe edge is not the best possible adjective here,
but it certainly isn't the one killer app that requires the whole
uid/gid scheme needing to be locked down.

People running into NFS uid/gid problems can figure out their own ways
to deal with it, (and that doesn't always imply mapping everything to
root plus norootsquash...)

> Package a few things in Gentoo that need a UID and/or GID and you will start 
> to understand the problem from a operating system packager perspective.

I have packaged a few things in Gentoo (privately only), and written
more shell installers, puppet manifests, ansible playbooks and user
account deployers than I care to recall; I've never run into this
problem that I couldn't solve trivially - usually by just knowing the
username|groupname and looking up the corresponding uid/gid. Really,
it's just data mapping and we have tools to do the lookup real fast.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 4714 bytes --]

On Wednesday, November 30, 2016 12:49:44 AM EST Alan McKinnon wrote:
> 
> Why would you end up with duplicated UIDs and GIDs? The only real ways
> that can happen is
> - ebuild "edits" passwd and group directly using echo/sed and the like.
> - ebuild runs useradd|groupadd specifying the uid/gid as arguments

I think you mean enewgroup and enewuser

> Both of which are silly. Just use useradd/groupadd without uid/gid
> arguments. The utility will make sure the uid/gids are non-duplicate,
> and ensure they are <1000 or whatever for system accounts

Randomly chosen GID and UID are a problem in the making. If you haven't 
experienced such yet, give yourself time. Moving files between systems, you 
have to chown/chgrp, etc it is NOT fun...

Or worse you mix stuff and give something improper permissions and really mess 
up security...

> How do you intend to MAKE devs follow it? More eternal bike-shedding?

A nifty tool called repoman which could do a quick lookup. As could enewgroup/
enewuser. They could hit the list/database. If something is trying to use 
existing error, etc. Otherwise process to reserve it, etc.
 

> Who cares what the uid/gid is? There's a range of about 950 to chose
> from. The way to ensure a filesystem object has the correct owner and
> group is by using chown/chgrp.

See above, any administrator moving files between systems, restoring backups, 
etc.

Say you do a fresh install. What if all your UID/GID differ from your backup? 
HUGE MESS!!!!
 
> Except for a few cases out on left field (like nfs shares - a problem
> that nfs must fix) you don't really care what the uid/gid is, as long as
> it's not duplicated. The thing you care about is the NAME

Not really just cases you haven't run into yet, which can be very common.
 
> > This is not needless bureaucracy , this is necessary.
> 
> This is a joke right?

Not at all, others are clearly not aware of all the potential issues, having 
not experienced them first hand, yet....

Work with enough systems, move files around, share lots of stuff, restore 
backups, you will start to see a major need.

> >> Have you watched the IANA ports assignment registry over the years?
> >> Consider how many services and tools you've seen that *don't* respect
> >> it.
> > 
> > Yes, how often to ports < 1024 change? Hardly ever.... Proving the exact
> > point why this is needed. People can change them themselves but 99% of
> > the time its to some other port > 1024.
> > 
> > Why is there IANA port assignment registry in the first place? Likely for
> > a
> > similar reason.
> 
> It's so that things like browsers, email tools and the like can drop
> 
> :<port> for the most part and be reasonably sure stuffs will still work.
> 
> Of the 65535 +-1 possible port numbers, only the first 1024 are truly
> important, and of those less than about a quarter are in common use
> (wild guess).

Most of the UID/GID I speak of are below 1000. System accounts, daemons, etc. 
Very likely the exact same stuff running on privileged ports  but not all.

> The next 10,000 or so are not standards by any means, just a list of
> stuff that happens to have been seen in the wild. Apps can and do pick
> any old port they feel like - witness the several things that will use
> 5000 out the box. Is this a problem? Not really, as very very few
> machines out there will install two apps both trying to use port 5000 by
> default.

Nor would that ever be with any system. All *nix systems have a reserved UID/
GID range and users stuff starts above that. Some 500, others 1000, etc.


> I have packaged a few things in Gentoo (privately only)

Try doing it for the public, which will end up with thousands of installs.

> , and written
> more shell installers, puppet manifests, ansible playbooks and user
> account deployers than I care to recall; I've never run into this
> problem that I couldn't solve trivially - usually by just knowing the
> username|groupname and looking up the corresponding uid/gid. Really,
> it's just data mapping and we have tools to do the lookup real fast.

Clearly you haven't come across it yet, and likely because experience has 
differed. But I have given you a few examples of how this could happen to 
anyone and why there would be a need.

Say it is a failed mail server, and you need to take the queue/spool to 
another. Same with print, or other jobs... You need them to have the same UID/
GID, or you end up wasting MORE time syncing them to the system they go onto. 
Much easier to ensure all are the same.

This goes for many other things. Lots of data gets owned by system accounts. 
Moving that data from system to system, with different UID/GIDs is a 
nightmare...

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Alan McKinnon]

On 30/11/2016 01:13, William L. Thomson Jr. wrote:
> On Wednesday, November 30, 2016 12:49:44 AM EST Alan McKinnon wrote:
>>
>> Why would you end up with duplicated UIDs and GIDs? The only real ways
>> that can happen is
>> - ebuild "edits" passwd and group directly using echo/sed and the like.
>> - ebuild runs useradd|groupadd specifying the uid/gid as arguments
> 
> I think you mean enewgroup and enewuser

Or that. Don't fixate on the tool or it's name, the question is about
your stateent of duplicate UIDs and GIDs. I responded with "how would
that occur?" and gave 2 scenarios I could think off of the bat

> 
>> Both of which are silly. Just use useradd/groupadd without uid/gid
>> arguments. The utility will make sure the uid/gids are non-duplicate,
>> and ensure they are <1000 or whatever for system accounts
> 
> Randomly chosen GID and UID are a problem in the making. If you haven't 
> experienced such yet, give yourself time. Moving files between systems, you 
> have to chown/chgrp, etc it is NOT fun...

chown/chgrp/chmod is not a chore. It is REQUIRED when moving files
around from system to system. Aside from the possibility of uid/gid
differing there is also umask in the mix which pretty much forces you to
chmod anyway. If you need to do that you might as well chown/chgrp as
well to be 100% sure.

I routinely and often do all the above at work, moving files between
FreeBSD, Ubuntu, Centos, Solaris and whatever container technology
du-jour the devs decided was teh coolz awes0me today. And powershell on
Unix plus posix/cygwin/bash on Windows. I have zero guarantees about any
id consistency in that mess^Wreal-life-ISP-network, so what difference
is Gentoo going to make?

> 
> Or worse you mix stuff and give something improper permissions and really mess 
> up security...

That's what chmod and eyeballs are for

> 
>> How do you intend to MAKE devs follow it? More eternal bike-shedding?
> 
> A nifty tool called repoman which could do a quick lookup. As could enewgroup/
> enewuser. They could hit the list/database. If something is trying to use 
> existing error, etc. Otherwise process to reserve it, etc.

OK, so you could force code into repoman to do what you want. You still
have to convince all other devs that it's a good idea
>  
> 
>> Who cares what the uid/gid is? There's a range of about 950 to chose
>> from. The way to ensure a filesystem object has the correct owner and
>> group is by using chown/chgrp.
> 
> See above, any administrator moving files between systems, restoring backups, 
> etc.

See above about what I do on a real life network. Daily. For 15 years
now and counting.

> 
> Say you do a fresh install. What if all your UID/GID differ from your backup? 
> HUGE MESS!!!!

So deal with it. That's what chmod is for.

Or do you reinstall, restore backups and then NOT check all details
thoroughly?

>  
>> Except for a few cases out on left field (like nfs shares - a problem
>> that nfs must fix) you don't really care what the uid/gid is, as long as
>> it's not duplicated. The thing you care about is the NAME
> 
> Not really just cases you haven't run into yet, which can be very common.

Describe these cases please. I've got 15 years behind me at this and
probably seen it all, but I'm just not seeing the problem you perceive

>  
>>> This is not needless bureaucracy , this is necessary.
>>
>> This is a joke right?
> 
> Not at all, others are clearly not aware of all the potential issues, having 
> not experienced them first hand, yet....
> 
> Work with enough systems, move files around, share lots of stuff, restore 
> backups, you will start to see a major need.

See above. I submit this problem does not really exist, that the real
solution is chmod/chown/chgrp/attention and eyeballs. Which you pretty
much have to do anyway for other reasons than lack of consistency around
UID/GID. Like umask, SGID on directories

[ snip ]

>> I have packaged a few things in Gentoo (privately only)
> 
> Try doing it for the public, which will end up with thousands of installs.

I'm still not seeing it. You can have a million installs from an ebuild
available to the public, and the vast majority are completely isolated
from each other.

> 
>> , and written
>> more shell installers, puppet manifests, ansible playbooks and user
>> account deployers than I care to recall; I've never run into this
>> problem that I couldn't solve trivially - usually by just knowing the
>> username|groupname and looking up the corresponding uid/gid. Really,
>> it's just data mapping and we have tools to do the lookup real fast.
> 
> Clearly you haven't come across it yet, and likely because experience has 
> differed. But I have given you a few examples of how this could happen to 
> anyone and why there would be a need.
> 
> Say it is a failed mail server, and you need to take the queue/spool to 
> another. Same with print, or other jobs... You need them to have the same UID/
> GID, or you end up wasting MORE time syncing them to the system they go onto. 
> Much easier to ensure all are the same.

Let's look at real life.

I have exim mail servers on FreeBSD, Ubuntu and Centos. Now that's a
typical real-life scenario, and they are different deliberately.
Shuttling the mail spool from one machine to another is a fairly lengthy
scp/rsync operation followed by a very fast chmod & chown and daemon
start. The saving is maybe 2 seconds in something that takes many minutes.

How about my name servers? They are also a mix and the hosted domains
are in cvs. The bind user does not have a shell and root doesn't get to
cvs update.

Now these are typical ordinary everyday sysadmin actions, and trying to
register standard UIDs on Gentoo does nothing to shave seconds off the
full process. So why bother? It would be optimizing for the less common
case.

I could also describe the enormous collection of jar files the devs want
dropped into new tomcat installs from any of several source code repos.

> This goes for many other things. Lots of data gets owned by system accounts. 
> Moving that data from system to system, with different UID/GIDs is a 
> nightmare...

I'm starting to think that you need more experience with real-world
deployments with things other than Gentoo in them. Your position only
affects Gentoo and does nothing to assist with the same circumstance on
everything that is not Gentoo - the common case.

The real problem here is not inconsistency, because that is something
you will never fix. The problem is the design of Unix itself and what
goes into inodes - a 16 bit number that maps to a name for the benefit
of humans. Syncing files between machines was never really envisaged -
humans worked with named and the IDs were whatever passwd said they
were. So as long as you picked a UID and stuck with it, all was good.
So you have a system where the UID can be any 16 bit number you want it
to be, and THAT is the real problem - people DID make it anything they
wanted it to be. That genie is long since out of the bottle and isn't
going back in.


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 2553 bytes --]

First off this is a Gentoo development list. I am talking about a problem and 
solution 100% Gentoo specific. Other OS, even Linux distros really are not 
relevant really to the discussion. Completely off topic to the thread.

On Wednesday, November 30, 2016 3:18:02 AM EST Alan McKinnon wrote:
>
> I routinely and often do all the above at work, moving files between
> FreeBSD, Ubuntu, Centos, Solaris and whatever container technology
> du-jour the devs decided was teh coolz awes0me today. And powershell on
> Unix plus posix/cygwin/bash on Windows.

You are describing a completely different scenario and problem. I am seeking 
to address something Gentoo specific.

I am also coming from the perspective of running fleets of Gentoo systems. 
Where some are brought online and offline at various intervals. You have systems 
automatically syncing things between them via rsync, dsync, and other. Trying 
to toss in chown/chgrp into the mix is not feasible nor necessary.

> Describe these cases please. I've got 15 years behind me at this and
> probably seen it all, but I'm just not seeing the problem you perceive

16 almost 17 here, but lets not get into pissing contests, there is little 
benefit or point.

> Let's look at real life.
> 
> I have exim mail servers on FreeBSD, Ubuntu and Centos. Now that's a
> typical real-life scenario, and they are different deliberately.

I am running ALL Gentoo, and Dovecot with dsync replication between systems.

> Shuttling the mail spool from one machine to another is a fairly lengthy
> scp/rsync operation followed by a very fast chmod & chown and daemon
> start. The saving is maybe 2 seconds in something that takes many minutes.

Not really, I can move qmail queues rather easily...

> How about my name servers? They are also a mix and the hosted domains
> are in cvs. The bind user does not have a shell and root doesn't get to
> cvs update.

Once again ALL Gentoo and BIND

> I'm starting to think that you need more experience with real-world
> deployments with things other than Gentoo in them. 

While I run a ALL Gentoo environment I have worked with others. My experience 
and background is not relevant, nor is yours and such is off topic. Anything 
other than Gentoo does not really matter on this list.

> Your position only
> affects Gentoo and does nothing to assist with the same circumstance on
> everything that is not Gentoo - the common case.

I am discussing concerns related to UID/GID management on Gentoo systems on a 
Gentoo development list...

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1692 bytes --]

On Tue, 29 Nov 2016 18:13:29 -0500
"William L. Thomson Jr." <wlt-ml@o-sinc.com> wrote:

> On Wednesday, November 30, 2016 12:49:44 AM EST Alan McKinnon wrote:
> > 
> > Why would you end up with duplicated UIDs and GIDs? The only real ways
> > that can happen is
> > - ebuild "edits" passwd and group directly using echo/sed and the like.
> > - ebuild runs useradd|groupadd specifying the uid/gid as arguments  
> 
> I think you mean enewgroup and enewuser

FYI, enew* functions handle UID/GID collisions gracefully, and just
fallback to using next free UID/GID.

> > Who cares what the uid/gid is? There's a range of about 950 to chose
> > from. The way to ensure a filesystem object has the correct owner and
> > group is by using chown/chgrp.  
> 
> See above, any administrator moving files between systems, restoring backups, 
> etc.
> 
> Say you do a fresh install. What if all your UID/GID differ from your backup? 
> HUGE MESS!!!!

I'm not sure if you're aware that but most of tools doing backups
actually use usernames/group names. So does new enough tar. So does
ssh.

Are you specifically using some obsolete or braindead tools to prove
your point? If you don't sync UIDs/GIDs properly, then you don't use
them when moving data across systems. Simple as that.

The only thing that you could worry about then are missing users/groups
on the target system. But then, so far none of your talk solved that
problem.

Furthermore, I should add that neither repeating the same argument
thrice, nor adding some random caps and exclamations marks, won't make
it any more valid.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1461 bytes --]

On Wednesday, November 30, 2016 8:54:42 AM EST Michał Górny wrote:
> On Tue, 29 Nov 2016 18:13:29 -0500
> 
> "William L. Thomson Jr." <wlt-ml@o-sinc.com> wrote:
>>
> > I think you mean enewgroup and enewuser
> 
> FYI, enew* functions handle UID/GID collisions gracefully, and just
> fallback to using next free UID/GID.

I would disagree with such and some what makes specifying a UID/GID pointless 
if it simply will use the next available in the event of a collision. Which 
available likely comes from the default allocation range > 500 or 1000. If 
system and was intended to be below that, not really ideal.
 
> I'm not sure if you're aware that but most of tools doing backups
> actually use usernames/group names. So does new enough tar. So does
> ssh.

tar can map users and groups via file, but why waste the time with such?

> Are you specifically using some obsolete or braindead tools to prove
> your point? If you don't sync UIDs/GIDs properly, then you don't use
> them when moving data across systems. Simple as that.

I start with consistent base images and have the same uid/gid all on all so 
syncing is not needed. Nor do I need to deal with it during restoration.

> The only thing that you could worry about then are missing users/groups
> on the target system. But then, so far none of your talk solved that
> problem.

A problem that should not exist with a proper setup.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 6545 bytes --]

On Tuesday, November 29, 2016 04:49:24 PM William L. Thomson Jr. wrote:
> On Tuesday, November 29, 2016 10:40:20 AM EST Michael Mol wrote:
> > Highly detailed lists like that--used as a broad standard--are a bad idea.
> > They represent a single synchronization point that everyone must adhere
> > to.
> 
> That is a statement based on opinion.

Of course. And then I gave examples as to why.

> You say it is a bad idea. I say it is
> necessary and needed. Otherwise wrt to Gentoo ebuilds can stomp on each
> other. Using same GID or UID in more than one ebuild causing problems.
> There has to be something know so others do not use ones others are
> already.

If Gentoo wants to do it internally, that's one thing. But I would recommend 
against inviting other distributions to use Gentoo's list, which was something 
you seemed to be suggesting. Doing so asks that Gentoo shoulder the 
bureaucratic load from other distributions that want things added to Gentoo's 
list.

> 
> > That means that every prospective adjustment to the list requires active
> > maintenance. That means that for every new daemon someone writes, they
> > have
> > to go through an admissions process. For every contentious fork of a
> > project, you risk conflict over who the designated contact for the
> > assignment should be.
> 
> If they package such in Gentoo someone is making a call as to what UID and
> GID should be used. If you think about it from packaging said new daemon in
> Gentoo, it is a MUST.
> 
> If it does not exist, should it be entirely random from the packager
> perspective? What if they use a GID/UID specific to them and not others.
> 
> There has to be some standard some consistency in Gentoo.

If you want to tie this specifically to Gentoo packaging, that's fine. Though 
I'd recommend you put the user and group allocation in the ebuild. Then your 
"list" is trivially generable by parsing portage. Further, you can *enforce* 
these allocations when calculating the dependency tree. If you're not 
enforcing them, what's the point? Is there a benefit without said enforcement?

> 
> > It adds a large bureaucratic load on everyone. Every itch some developer
> > thinks about scratching has to be weighed against engaging with some
> > process- laden entity. Maybe they'll participate, but they likely won't.
> 
> Gentoo shines at bureaucratic load. That may be one of the only things
> Gentoo is really good at, needless bureaucratic loads that just slow things
> down and fracture the community, exherbo, funtoo, and likely others...

I was under the impression that Gentoo was chronically undermanned for even 
the workload it has.

> 
> This is not needless bureaucracy , this is necessary.

Opinion. Why is it necessary? What is it necessary for?

> 
> > Have you watched the IANA ports assignment registry over the years?
> > Consider how many services and tools you've seen that *don't* respect it.
> 
> Yes, how often to ports < 1024 change? Hardly ever.... Proving the exact
> point why this is needed. People can change them themselves but 99% of the
> time its to some other port > 1024.
> 
> Why is there IANA port assignment registry in the first place? Likely for a
> similar reason.

How relevant even *is* the <1024 distinction any longer? Once upon a time, the 
idea was you had to have special privileges to open those ports. Now, there is 
really no reason for anyone to care; capabilities-oriented permissions 
completely obviated the need, and I can only think of ssh, telnet and ftp and 
as server services that should require special host privileges to 
operate...and that's only because they may need to be able to call setuid().

And because the <1024 port privilege distinction has been so restrictive and 
bureaucratically sloggish, applications adapted to use ports above 1024. 
Games? Sync utilities? Proxy servers? Far more commonly-observed ports are 
above 1024 than below it, and many (most?) don't even get added to IANA's 
list. *That's* why the <1024 ports don't change much; the feature is obsolete, 
and users don't bother.

As an example, I just checked on Syncthing, to see if its three ports were on 
IANA's list. They're not, and I stumbled across a Github issue where the devs 
flatly stated they didn't care.

The IANA ports list is, by and large, obsolete. It became obsolete because it 
was too much a hassle for people to participate in.

> 
> > All of this is why we use identity management tools like LDAP in the first
> > place. Heck, it's why we have passwd and group files for mapping names to
> > ids and didn't simply hardcode system IDs decades ago.
> 
> LDAP typical manages user accounts not system. If the LDAP server is not
> reachable you would make a system completely nonfunctional if it relied on
> LDAP for system accounts.

That's fair. Although I really like how one LDAP alternative operates over 
DNS, permitting local caching. (I can't for the life of me remember the name 
of that system, though.)

> 
> Also needed from a file sharing stand point of view if sharing parts of a
> system across others. You need consistent GID/UID mappings or things like
> NFS will have lots of problems.
> 
> Package a few things in Gentoo that need a UID and/or GID and you will start
> to understand the problem from a operating system packager perspective.

Oh, I understand the problem, but you haven't explained why your solution is 
the necessary solution to it, or how you would cope with the plethora of edge 
cases I brought up. It would seem there are already many established 
workarounds for the status quo, unstable-UID/GID in a cross-system context.

Now, would I like to see stable UIDs and GIDs? Sure. For a couple of years, 
I've been toying with the idea of having IPSEC AH packets tagging packets with 
the UID of the process that generated them, for diagnostic and auditing 
purposes. Stable UIDs would make that more useful.

But trying to set up a list for everyone to move in lockstep with seems to me 
like a bad way to go. Less bad if you intend to keep it unique to Gentoo, but 
the broader you make the scope, the more strain you'll put on the ecosystem as 
a whole. More daemons will be build that are intended to run as local users. 
More software will be pushed into opaque blobs a la Snap and Flatpack.

As a general rule, the bigger the hassle you make something, the less people 
will want to engage.


-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 3825 bytes --]

On Wednesday, November 30, 2016 1:22:07 PM EST Michael Mol wrote:
>
> 
> If Gentoo wants to do it internally, that's one thing. 

This list is about Gentoo internal things

> But I would recommend
> against inviting other distributions to use Gentoo's list, which was
> something you seemed to be suggesting. Doing so asks that Gentoo shoulder
> the bureaucratic load from other distributions that want things added to
> Gentoo's list.

Gentoo cannot force others to do anything. If Gentoo is leading in a 
direction, others choose to follow or not. Gentoo does not set standards that 
would be up to LSB and/or POSIX.

My point is Gentoo should do its own thing, lead the way. Ideally others 
follow and it becomes a standard either in LSB or POSIX. Hopefully that will 
clarify my position.

> If you want to tie this specifically to Gentoo packaging, that's fine.

Which is why it is being discussed on a Gentoo development list and not 
others.

> Though I'd recommend you put the user and group allocation in the ebuild.
> Then your "list" is trivially generable by parsing portage. Further, you
> can *enforce* these allocations when calculating the dependency tree. If
> you're not enforcing them, what's the point? Is there a benefit without
> said enforcement?

As stated, enewuser/enewgroup would utilize such a list/database directly. In 
addition to repoman so issues are prevented before ebuilds are committed.

> > This is not needless bureaucracy , this is necessary.
> 
> Opinion. Why is it necessary? What is it necessary for?

It is necessary so Gentoo base system installs are consistent from one system 
to the next, Just as RHEL and Debian, and likely others. When working with 
large amounts of installs, You want them all to be the same or as close to 
identical as possible. Thus the rise of Docker, CoreOS, etc.
 
> Oh, I understand the problem, but you haven't explained why your solution is
> the necessary solution to it, or how you would cope with the plethora of
> edge cases I brought up. It would seem there are already many established
> workarounds for the status quo, unstable-UID/GID in a cross-system context.

My solution is to avoid such issues. I start with a common base image. I try 
to ensure anything else installed beyond that, which adds new users/groups is 
the same. At times I will re-image and use that as well for other similar 
systems. Rather than mess with doing the same install to many and trying to 
sync UID/GID.

Think cloning rather than installing.

> But trying to set up a list for everyone to move in lockstep with seems to
> me like a bad way to go.

See my other post, other distros already do this for core system accounts.

> Less bad if you intend to keep it unique to
> Gentoo, but the broader you make the scope, the more strain you'll put on
> the ecosystem as a whole. 

Standards need to exist so there is consistency. In the absence of said 
standard, next best thing you can do is look to what others are doing and do 
the same. Thus I tend to say go with RedHat UID/GID over say Arch, maybe even 
Debian.   But those two likely have larger install bases than most any other 
distro. If the UID/GID are the same between RedHat and Debian, that already 
makes a good deal of systems consistent now.

> More daemons will be build that are intended to
> run as local users. More software will be pushed into opaque blobs a la
> Snap and Flatpack.

I am talking about core system accounts

> As a general rule, the bigger the hassle you make something, the less people
> will want to engage.

When standards exist, others will follow, ideally. When standards do not 
exist, everyone is left to their own way of doing things. IMHO it is less of a 
hassle to comply with standards than all the various ways of doing something.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 6214 bytes --]

On Wednesday, November 30, 2016 01:41:24 PM William L. Thomson Jr. wrote:
> On Wednesday, November 30, 2016 1:22:07 PM EST Michael Mol wrote:
> > If Gentoo wants to do it internally, that's one thing.
> 
> This list is about Gentoo internal things

Here, let me bring up a bit of recent history from your Message-ID 
<assp.0140865882.25530652.rRlbQJgv4Y@wlt>, which had a signature of 
iEYEABECAAYFAlg8iAQACgkQTXGypIOqM1A2EgCglmZkNYaJ16qQkSxezTqCtI4/
PwoAnR2dW0XUFZk8QUmgrVwu+3OpRxS+
=tuat, which my client indicated matched the key 
0xC47A576A663995BADF1B54724D71B2A483AA3350, but I don't have your key trusted, 
so whatever:

> I believe the main reason such is the case is a lack of any such list or 
> database for others to adhere to. Once again an area Gentoo could be
> leading. 
> Had Gentoo done this years ago others might have adopted.

> IMHO it is something that should be  a part of LSB. If not POSIX in general. 
> One cannot really change the past or current state of things. But can make 
the future better.

> For now who cares about other OS or distros. If Gentoo gets its house in
> order 
> others may follow.

I will note that it's this point when I first replied; that was the point when 
you chose to expand the scope outside Gentoo.

> 
> > But I would recommend
> > against inviting other distributions to use Gentoo's list, which was
> > something you seemed to be suggesting. Doing so asks that Gentoo shoulder
> > the bureaucratic load from other distributions that want things added to
> > Gentoo's list.
> 
> Gentoo cannot force others to do anything.

I didn't say force. I said invite.

> If Gentoo is leading in a
> direction, others choose to follow or not. Gentoo does not set standards
> that would be up to LSB and/or POSIX.
> 
> My point is Gentoo should do its own thing, lead the way. Ideally others
> follow and it becomes a standard either in LSB or POSIX. Hopefully that will
> clarify my position.

As you noted, Arch appeared to attempt this, and others did not follow.

> 
> > If you want to tie this specifically to Gentoo packaging, that's fine.
> 
> Which is why it is being discussed on a Gentoo development list and not
> others.

That's fine. As I pointed out, I only started chiming in when you began 
advocating exporting Gentoo's list to a broader ecosystem.

[snip]

> > > This is not needless bureaucracy , this is necessary.
> > 
> > Opinion. Why is it necessary? What is it necessary for?
> 
> It is necessary so Gentoo base system installs are consistent from one
> system to the next, Just as RHEL and Debian, and likely others. When
> working with large amounts of installs, You want them all to be the same or
> as close to identical as possible. Thus the rise of Docker, CoreOS, etc.

If RHEL and Debian are consistent from one system to the next, obviously it's 
sensical to use their list. But why don't they use each others? Or am I 
missing something, and that's exactly what they're doing?

> 
> > Oh, I understand the problem, but you haven't explained why your solution
> > is the necessary solution to it, or how you would cope with the plethora
> > of edge cases I brought up. It would seem there are already many
> > established workarounds for the status quo, unstable-UID/GID in a
> > cross-system context.
> My solution is to avoid such issues. I start with a common base image. I try
> to ensure anything else installed beyond that, which adds new users/groups
> is the same. At times I will re-image and use that as well for other
> similar systems. Rather than mess with doing the same install to many and
> trying to sync UID/GID.
> 
> Think cloning rather than installing.

Sure. But if you clone a seed node, does it matter that a second from-scratch 
install may not have the same mapping?

[snip]

> > Less bad if you intend to keep it unique to
> > Gentoo, but the broader you make the scope, the more strain you'll put on
> > the ecosystem as a whole.
> 
> Standards need to exist so there is consistency. In the absence of said
> standard, next best thing you can do is look to what others are doing and do
> the same. Thus I tend to say go with RedHat UID/GID over say Arch, maybe
> even Debian.   But those two likely have larger install bases than most any
> other distro. If the UID/GID are the same between RedHat and Debian, that
> already makes a good deal of systems consistent now.

If UID/GID are consistent between RH and Debian, then yeah, what you have is a 
de facto standard, and it would be reasonable to conform, if there are people 
who actually have a need for that cross-system mirroring.

> 
> > More daemons will be build that are intended to
> > run as local users. More software will be pushed into opaque blobs a la
> > Snap and Flatpack.
> 
> I am talking about core system accounts

Who decides what qualifies as a core system account?

If there's any trend I've been able to clearly observe over the last fifteen 
years, it's the grinding of such boundaries into finer and finer granularity. 
Heck, I think there was a thread on gentoo-user some time in the last few 
months where someone wanted to be able to use two different MTAs on the same 
host! (Obviously, he couldn't, but he had a use case.)

Heck, some time five or six years ago, I filed a bug report asking that some 
core package (maybe it was gcc?) have its build dependencies properly defined. 
I was told that wasn't going to happen, as doing that for all the core 
packages would be too difficult or some such; their dependencies would be left 
coarse. And now we've had threads in the last few months touching on resolving 
that very thing.

> 
> > As a general rule, the bigger the hassle you make something, the less
> > people will want to engage.
> 
> When standards exist, others will follow, ideally. When standards do not
> exist, everyone is left to their own way of doing things. IMHO it is less of
> a hassle to comply with standards than all the various ways of doing
> something.

For the packager, for sure. For developers trying to make new things, not so 
much.

-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 3562 bytes --]

On Wednesday, November 30, 2016 3:08:30 PM EST Michael Mol wrote:
>
> > IMHO it is something that should be  a part of LSB. If not POSIX in
> > general. One cannot really change the past or current state of things.
> > But can make
> the future better.
> 
> > For now who cares about other OS or distros. If Gentoo gets its house in
> > order
> > others may follow.
> 
> I will note that it's this point when I first replied; that was the point
> when you chose to expand the scope outside Gentoo.

Stop making things into something they are not. Re-read the above I said it 
should be part of official standards. I also said others MAY follow...

> > Gentoo cannot force others to do anything.
> 
> I didn't say force. I said invite.

I never typed the word invite. I never mentioned Gentoo being proactive about 
pushing its specific things on others. Please stop making stuff up and going 
way off topic.
 
 
> As you noted, Arch appeared to attempt this, and others did not follow.

Arch themselves never got it squared away. It was just a concept. If Arch does 
not implement it how can others? I hardly consider Arch a leading distro like 
RHEL or Debian, which both have derivatives in wide use, Fedora, CentOS and 
Ubuntu.

That right there likely covers over 50% of all Linux installs.


> That's fine. As I pointed out, I only started chiming in when you began
> advocating exporting Gentoo's list to a broader ecosystem.

You are reading things I never typed, and coming up with some far fetched 
scenarios. Nothing you are saying is anywhere near what I wrote.

> If RHEL and Debian are consistent from one system to the next, obviously
> it's sensical to use their list. But why don't they use each others? Or am
> I missing something, and that's exactly what they're doing?

Going back to my first point about this being part of LSB or POSIX. Because it 
is part of neither RedHat and Debian do things differently.

Why does RedHat not use deb format over rpm. Why does Debian use deb instead 
of RPM. These are different distros with different approaches. If their UID/
GID are the same, its likely per legacy reasons. Though they may be looking at 
each other.

Debian at this time does not produce a list. The only I found were RedHat and 
Arch, with Archs' being unofficial and never adopted.

> Sure. But if you clone a seed node, does it matter that a second
> from-scratch install may not have the same mapping?

Yes if they are to be added to the same fleet or cluster of systems. In that 
event it would likely start a new from scratch base image. But that is pretty 
rare. I do update base images, though rarely do system UID/GID change from 
initial install.

> If UID/GID are consistent between RH and Debian, then yeah, what you have is
> a de facto standard, and it would be reasonable to conform, if there are
> people who actually have a need for that cross-system mirroring.

If Gentoo does the same, that would make one other and moving all more in the 
direction of a standard.

> > > More daemons will be build that are intended to
> > > run as local users. More software will be pushed into opaque blobs a la
> > > Snap and Flatpack.
> > 
> > I am talking about core system accounts
> 
> Who decides what qualifies as a core system account?

This is pretty silly now and way off topic. I will leave it to others to 
decide. I would prefer to go beyond just system so it is Gentoo wide. Arch was 
not limited to system stuff, like RedHat and Debian.

Really up to Gentoo Developers to decide it all.

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 7062 bytes --]

On Wednesday, November 30, 2016 03:25:21 PM William L. Thomson Jr. wrote:
> On Wednesday, November 30, 2016 3:08:30 PM EST Michael Mol wrote:
> > > IMHO it is something that should be  a part of LSB. If not POSIX in
> > > general. One cannot really change the past or current state of things.
> > > But can make
> > 
> > the future better.
> > 
> > > For now who cares about other OS or distros. If Gentoo gets its house in
> > > order
> > > others may follow.
> > 
> > I will note that it's this point when I first replied; that was the point
> > when you chose to expand the scope outside Gentoo.
> 
> Stop making things into something they are not. Re-read the above I said it
> should be part of official standards. I also said others MAY follow...

Honestly, that sounded to me like advocacy; "a benefit of doing this is that 
others may follow." If that's not the spirit in which it was intended, I 
apologize.

> 
> > > Gentoo cannot force others to do anything.
> > 
> > I didn't say force. I said invite.
> 
> I never typed the word invite. I never mentioned Gentoo being proactive
> about pushing its specific things on others. Please stop making stuff up
> and going way off topic.

As I note above, I interpreted what you said as advocacy.

> 
> > As you noted, Arch appeared to attempt this, and others did not follow.
> 
> Arch themselves never got it squared away. It was just a concept. If Arch
> does not implement it how can others? I hardly consider Arch a leading
> distro like RHEL or Debian, which both have derivatives in wide use,
> Fedora, CentOS and Ubuntu.
> 
> That right there likely covers over 50% of all Linux installs.
> 
> > That's fine. As I pointed out, I only started chiming in when you began
> > advocating exporting Gentoo's list to a broader ecosystem.
> 
> You are reading things I never typed, and coming up with some far fetched
> scenarios. Nothing you are saying is anywhere near what I wrote.

Again, read above. If that's not how it was intended, I apologize.

> 
> > If RHEL and Debian are consistent from one system to the next, obviously
> > it's sensical to use their list. But why don't they use each others? Or am
> > I missing something, and that's exactly what they're doing?
> 
> Going back to my first point about this being part of LSB or POSIX. Because
> it is part of neither RedHat and Debian do things differently.


You're asserting that Red Hat and Debian do things differently because there's 
nobody to force them to do things the same way. It can't be because there's no 
reference for them to look at; for sure, the second into market could simply 
have looked at the first. It's probable they did.

I know Debian starts their non-system UIDs at 1000, while RH, once upon a 
time, started theirs at 500. Why the difference? Dunno. RH came before Debian, 
so I imagine Debian wanted a bit more headroom to work with. Are there static 
UIDs in the 500-999 range on Debian? That would be why RH doesn't use Debian's 
set; they'd have a UID conflict on their hands.

Staring at a CentOS7 live environment in front of me, it looks like RH now 
starts at 1000.

It's probable they could settle on a common spec now, but there would still be 
a great number of legacy systems out there to support., and you've still got a 
very limited namespace to work with.

> 
> Why does RedHat not use deb format over rpm. Why does Debian use deb instead
> of RPM. 

Well, RPM was developed to be a better alternative to the tarball. Debian 
thought the RPM format was lacking, and developed their own spec. For sure, 
nobody likes to do work for no reason. Even hugely disruptive changes have 
motivations behind them.

I'm sorry, was that a rhetorical question? I just realized...

> These are different distros with different approaches. If their
> UID/ GID are the same, its likely per legacy reasons. Though they may be
> looking at each other.
> 
> Debian at this time does not produce a list. The only I found were RedHat
> and Arch, with Archs' being unofficial and never adopted.

I'll note I'm treating the concept of a list as very abstract; if things are 
consistent, then there's de facto a consistent state that could be distilled 
deterministically into a listing.

> 
> > Sure. But if you clone a seed node, does it matter that a second
> > from-scratch install may not have the same mapping?
> 
> Yes if they are to be added to the same fleet or cluster of systems. In that
> event it would likely start a new from scratch base image. But that is
> pretty rare. I do update base images, though rarely do system UID/GID
> change from initial install.

You know, I would expect for a system of that scale, that you'd have 
standardized and preseeded your passwd and group files with your site standard 
enumerations. It would be trivial to do in any Gentoo install; copy your files 
into place before your initial chroot. All of which you should have scripted 
at this point. If you'd like, I'll send you a link to mine; you can use it and 
adapt it for whatever purpose you need.

> 
> > If UID/GID are consistent between RH and Debian, then yeah, what you have
> > is a de facto standard, and it would be reasonable to conform, if there
> > are people who actually have a need for that cross-system mirroring.
> 
> If Gentoo does the same, that would make one other and moving all more in
> the direction of a standard.

You spent a thousand or so words telling me how other distros went about 
assigning UIDs for <user#, that hopefully if Gentoo standardized a list of 
assignments, other users would follow--while telling me this wasn't about 
other distros, but specifically about Gentoo, on a gentoo-specific list.

If you're as passionate about the problem as you seem, maybe you *should* be 
pushing LSB to engage Debian and Red Hat, or be the conduit for that 
engagement. Saying that Gentoo should lead by example in the hopes that 
someone else might make the effort for cross-distro consistency is...odd.

> 
> > > > More daemons will be build that are intended to
> > > > run as local users. More software will be pushed into opaque blobs a
> > > > la
> > > > Snap and Flatpack.
> > > 
> > > I am talking about core system accounts
> > 
> > Who decides what qualifies as a core system account?
> 
> This is pretty silly now and way off topic. I will leave it to others to
> decide. I would prefer to go beyond just system so it is Gentoo wide. Arch
> was not limited to system stuff, like RedHat and Debian.
> 
> Really up to Gentoo Developers to decide it all.

No, that leads to a very serious question of philosophy and ontology. And it's 
a hard question: What defines a core system account? What analytical test 
exists that can sanely provide for statically assigning 999 unique numbers 
such that a smartwatch, an access point, a web server and a virtualization 
host can live comfortably in such a small space?

-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1793 bytes --]

On Wednesday, November 30, 2016 4:38:57 PM EST Michael Mol wrote:
> 
> You're asserting that Red Hat and Debian do things differently because
> there's nobody to force them to do things the same way. It can't be because
> there's no reference for them to look at; for sure, the second into market
> could simply have looked at the first. It's probable they did.

I would recommend doing some research on LSB. If Debian gave up deb for RPM, 
then you would see more unification. Ultimately the two had different origins, 
purposes, and thus can only align on some things but not all.

You can only push standards so far. When you have two different opposing ways 
of doing something. A call has to be made as to which to standardize unless 
you do both. It is impossible to please everyone all the time when it comes to 
consensus stuff. People have different ideas and will accomplish the same 
things in different ways.

Look at the whole systemd thing that came out of RedHat and took over the 
Linux world. It damn near fractured Debian, and did create a fork. Most other 
distros including Debian gave in.

Gentoo is one of the few still offering an alternative OpenRC. Much like the 
UID/GID thing I blame Gentoo for OpenRC not being adopted more widely and 
giving room for things like systemd to take over with little alternative for 
most.

This kind of thing will happen again and again. Though if other distros do not 
start leading. It will become more and more a RedHat lead Linux world. Though 
some things go the other way. RedHat seems to have adopted Debians 
alternatives system, which is basically eselect, and related tools *-config.

But we are way off topic to the userkit.eclass and UID/GID. New thread, but I 
am past the discussion. Thanks!

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[William L. Thomson Jr.]

[-- Attachment #1: Type: text/plain, Size: 1168 bytes --]

A couple more links, I should have provided initially as they better support 
the argument.

First from Debian, I cannot find a list, but it is clearly mentioned.

"0-99:
Globally allocated by the Debian project, the same on every Debian system"
https://www.debian.org/doc/debian-policy/ch-opersys.html#s9.2.2

This is even better, what Gentoo lacks, and could build upon.
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/
Deployment_Guide/s1-users-groups-standard-users.html

Also carries to CentOS of course
https://www.centos.org/docs/5/html/Deployment_Guide-en-US/s1-users-groups-standard-users.html

Per previous links installing some RPMs that have fixed UID/GID will result in 
problems of other things are using it

"The vdsm user however is fixed to a UID of 36 and the kvm group is fixed to a 
GID of 36.
If UID 36 or GID 36 is already used by another account on the system then a 
conflict will arise during installation of the vdsm and qemu-kvm-rhev 
packages."

https://access.redhat.com/documentation/en-US/
Red_Hat_Enterprise_Virtualization/3.5/html/Installation_Guide/sect-
System_Accounts.html

-- 
William L. Thomson Jr.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[james]

On 11/30/2016 10:23 AM, William L. Thomson Jr. wrote:
> A couple more links, I should have provided initially as they better support
> the argument.
>
> First from Debian, I cannot find a list, but it is clearly mentioned.
>
> "0-99:
> Globally allocated by the Debian project, the same on every Debian system"
> https://www.debian.org/doc/debian-policy/ch-opersys.html#s9.2.2
>
> This is even better, what Gentoo lacks, and could build upon.
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/
> Deployment_Guide/s1-users-groups-standard-users.html
>
> Also carries to CentOS of course
> https://www.centos.org/docs/5/html/Deployment_Guide-en-US/s1-users-groups-standard-users.html
>
> Per previous links installing some RPMs that have fixed UID/GID will result in
> problems of other things are using it
>
> "The vdsm user however is fixed to a UID of 36 and the kvm group is fixed to a
> GID of 36.
> If UID 36 or GID 36 is already used by another account on the system then a
> conflict will arise during installation of the vdsm and qemu-kvm-rhev
> packages."
>
> https://access.redhat.com/documentation/en-US/
> Red_Hat_Enterprise_Virtualization/3.5/html/Installation_Guide/sect-
> System_Accounts.html

I appreciate all the discussion on uid-gid as it is central to cluster 
provisioning work.

Some Background::


My specific area of development is heterogeneous (hardware) gentoo 
clusters with a specific focus on "uni-kernels' (UK). I define UK as a 
minimized, optimized hardened kernel that are specifically tuned to a 
minimized and optimize framework for a specific problem or specific 
category of problem for High Performance Computing (HPC) needs. In fact 
the need to benchmark and compare a myriad of codes, such as openstack 
on RHEL vs a skinny gentoo solution, on the exact same hardware will 
necessitate provisioning from bare metal up to full stack online and 
thus require numerous boot cycles. uid/gid symmetry would be a keen 
component of to my solutions. One of the challenges I have not worked on 
yet, is a systematic and automated solution for a variety of uid-gid 
differences between the systems I need to test and compare.

I am not certain that an ebuild or PMS level solution will work for 
comparing images(canned solutions from various sources) to a minimized 
and optimized gentoo solution. Furthermore, I'd definitely appreciated 
any advice and templates/profiles/scripts/etc that facilitate the 
automation of uid/gid compatibility for as wide a variety of 
kernels+OS+framework at least within gentoo. Note: for me a 'framework' 
is vary similar to the world-file. On other distros, a framework is the 
sum of additional codes on top of a basic installation of that distro. 
Applicability to other major distros, such as *bunu, RH, debian, and 
arch derivatives would be keenly useful for my research and development 
needs. Furthermore, I believe that docker is just killing the cluster 
competition with uni-kernels and a minimized distro such as Alpine. This
is an embarrassment to Gentoo that docker+alpine is 'killing it' in a 
space that is natural for Gentoo to dominate, imho.


This is a complex issue, as most of what has already been posted to this 
thread are all impactfully true. So flexibility is paramount, imho.  In 
fact if there is a way, I'd suggest that a multitude of scenarios are 
supported to the point that for my work there could easily be hundreds 
of variants. The keyword, 'profiles' comes to mind, but that has 
additional connotations within gentoo. Surely a robust and automated way 
to deal with differences in uid/gid between differing systems (same 
distro or not) would be an excellent project. If this is or is not 
possible, regardless of whether other distros use this capability, it 
would certainly aid folks in migrating other systems
from different distros to gentoo; so that bring enormous value to gentoo 
as a distro.


More specifically::

One thing is for sure, uni-kernels are just killing 'canned cluster' 
solutions for specific types of problems, particular defined by HPC. I
strongly believe that all of that pioneering work on HPC clustering will 
definitely impact routine web/admin/processing venues, eventually.
A given organization will be able to find the optimal images for their 
needs and then easily migrate their needs to a wide variety of 
datacenters for peak or scale-up. Unikernels in a wide variety of forms,
will enable hybrid clusters and ease the migration of business, web and 
other needs between clusters, in a seamless fashion.

A robust and flexible way to automate, orchestrate (overused term I 
know) and provision thousands of  systems is desperately needed, imo,
and a tool to transparently handle uid/gid differences would be keen.

I want to thank you, for introducing this topic and I tremendously 
appreciate all of the comments folks are interjecting, even the terse 
comments from admins that need a way to 'turn off' these features.
Ultimately, CoreOS has an automated provisioning system which when 
combined with a robust and flexible uid/gid solution would be a 
fantastic pathway to enabling gentoo to once again dominate the 
clustering world (like it did over  a decade ago), via optimized 
uni-kernels::

https://blog.docker.com/2016/01/unikernel/


Additionally, for a deeper dive::
BGO: 5932218

or these links::
https://github.com/coreos/coreos-baremetal

https://coreos.com/blog/introducing-ignition.htm

https://github.com/coreos/ignition/blob/master/doc/getting-started.md

https://github.com/coreos/ignition/blob/master/doc/supported-platforms.md


Thanks,
James

Re: [gentoo-dev] RFC: Userkit.eclass

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

On Wed, 23 Nov 2016 09:44:33 +0100
Manuel Rüger <mrueg@gentoo.org> wrote:

> What happens if the ebuild wants to create multiple users/group?
> Currently, I want to ignore that case and focus on the 80% ebuilds that
> can profit from such an eclass.

You can solve that part quite easily really.

Just deem USERKIT_USER and USERKIT_GROUP to be basenames
for idenitifiers.


Then you'd have

   get_user == returns "${USERKIT_USER}"

   get_user "admin" == returns "${USERKIT_USER}-admin"

Its not a perfect solution, but its better than "we just forget about this"

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] RFC: Userkit.eclass

[Zac Medico]

On 11/23/2016 09:46 AM, Kent Fredric wrote:
> On Wed, 23 Nov 2016 09:44:33 +0100
> Manuel Rüger <mrueg@gentoo.org> wrote:
> 
>> What happens if the ebuild wants to create multiple users/group?
>> Currently, I want to ignore that case and focus on the 80% ebuilds that
>> can profit from such an eclass.
> 
> You can solve that part quite easily really.
> 
> Just deem USERKIT_USER and USERKIT_GROUP to be basenames
> for idenitifiers.
> 
> 
> Then you'd have
> 
>    get_user == returns "${USERKIT_USER}"
> 
>    get_user "admin" == returns "${USERKIT_USER}-admin"
> 
> Its not a perfect solution, but its better than "we just forget about this"

That's not flexible enough, because we don't have control over the
user/group naming scheme used by upstream. So, we have to support an
unbounded number of arbitrarily named users and groups.
-- 
Thanks,
Zac

Re: [gentoo-dev] RFC: Userkit.eclass

[Zac Medico]

On 11/23/2016 12:44 AM, Manuel Rüger wrote:
> My only concerns right now are:
> Where to store those ${CATEGORY}-${PN}_user and ${CATEGORY}-${PN}_group?
> One solution could be to have another eclass named userkit-data.eclass,
> which is empty by default and needs to be forked to an overlay and then
> use the eclass-override setting in repos.conf. Unfortunately this will
> cause a lot of md5-cache rewrites.
> Another solution would be portage/package.env or portage/env.

Just allow for people do define bashrc functions that override the
eclass behavior, and they'll have the flexibility to implement anything
they want, without having to override eclasses.
-- 
Thanks,
Zac