[gentoo-project] eudev project announcement

[gentoo-project] eudev project announcement

[Richard Yao]

[-- Attachment #1: Type: text/plain, Size: 8613 bytes --]

Dear Everyone,

	I am pleased to announce the Gentoo eudev project. Many of you already
know about the eudev project from early publicity that we had before
things were ready. Despite that, I hope to take advantage of the
official announcement to explain what we are doing, why we are doing it
and what it means for you. I have broken the announcement into
subsections, each with a title for ease of reading.

Why fork udev?

	Earlier this year, udev upstream was absorbed into systemd. udev often
breaks compatibility with older systems by depending upon recent Linux
kernel releases, even when such dependencies are avoidable. This became
worse after udev became part of systemd, which has jeopardized our
ability to support existing installations. The systemd developers are
uninterested in providing full support in udev to systemd alternatives.
These are problems for us and we have decided to fork udev to address them.

You are a Gentoo project. What does this mean?

	Gentoo as an organization is quite similar to github, although it is
exclusive to Gentoo developers. Our rules permit all Gentoo developers
have the ability to start a project and such projects are entitled to be
hosted on Gentoo infrastructure. This by no means constitutes official
endorsement by Gentoo's governing body and we have no authority to
dictate the future direction of Gentoo. We do have the ability to
provide an alternative to Gentoo users, which we fully intend to do.
	eudev will be by no means exclusive to Gentoo. We will handle bug
reports from users of other distribution in the same way that we handle
bug reports involving Gentoo. This will be much like other Gentoo-hosted
projects such as portage and OpenRC.

What is your project's license?

	The systemd developers were in the middle of a transition to the LGPL
from the GPL when we forked. We inherited the code in the middle of that
transition and we see no reason to pursue a different course. Therefore,
all future changes that we make to eudev will be available under the LGPL.

What are your project's goals?

	Our primary goal is to address the problems with systemd-udev that
caused us to fork it in the first place. In particular, we want better
compatibility with existing software such as OpenRC and Upstart, older
kernels, various toolchains and anything else required by users and
various distributions. We also want to minimize regressions and work
with developers of other distributions (and components used by them) to
address issues.

How will you minimize regressions?

	We intend to maintain HEAD in a releaseable state. All minor changes
require review from one eudev developer and all major changes require
review from two eudev developers. This does not include the author. In
addition, we intend to require commits to make logically independent
changes with descriptive commit messages to make regression hunting
easier when regressions do happen.
	These rules were not enforced at the beginning, but we are
transitioning toward enforcement. They will enter full effect once we
tag our first release candidate.

How do you intend to work with other distributions?

	We have our repository on github, which is known for easy
collaboration. If a distribution developer identifies a problem with
eudev, they can file an issue and we will do our best to resolve their
problem. If they wish to work with us to resolve it, we can talk in IRC
and they can also file pull requests. Provided that the changes are not
entirely unreasonable (e.g. pushing an init system into udev), we are
willing to work with authors of pull requests to get them into a
mergeable state. The only hard rule is that changes cannot break the
ability of existing systems to upgrade.
	We also plan to make an official mailing list, which will be hosted on
Gentoo infrastructure.

Will you make the boot process faster?

	We have ideas on how to make udevd faster. However, people usually only
notice the time that udevd takes when there is a bug and we are more
interested in fixing those bugs than we are in shaving milliseconds off
boot time. There are plenty of areas that could use attention by people
that are interested in a faster boot process before udev becomes one of
them. We consider things such as a reliable boot process to be more
important than speed and we are willing to subject users to the
additional few hundred milliseconds that failing to tweak things for
speed incurs.
	We are already dealing with regressions that the systemd developers
introduced in their attempt to make things faster and we consider fixing
them to be a priority. However, we would be happy to collaborate with
people willing to work on boot speed improvements to get them into a
mergeable state. We encourage people interested in speed improvements to
talk to us about how they could be done in a reasonable manner. It would
be premature to do it at this instant, but it would definitely be
something that could be done after we have tagged a few stable releases.

What did systemd break when trying to make things faster?

	A good example involves module loading. Previously, each module load
would incur a roughly 10 to 20 ms disk access latency and a 0.01 ms fork
overhead. This was partially masked by udev's ability to execute rules
asynchronously through fork(), which meant that multiple modules could
be read in parallel.
	The introduction of kmod eliminated the 0.01 ms fork() overhead, and
consequently required each module to be loaded sequentially. This
imposes an overhead of 10 to 20 milliseconds times the number of
modules, which is asymptotically worse than what it replaced. A feature
of kmod intended to address that placed all modules into a single file,
which would actually make things faster. However, none of our users use
it and all of our users would suffer from it.
	In addition, the manner in which kmod was integrated has implications
beyond speed regressions. The use of kmod by udev permits a buggy kernel
module (possibly interacting with bad hardware) to hang in module_init.
This causes udevd to hang, which prevents further rule processing. This
is a bad situation, but we feel that it is important to handle bad
situations in a graceful manner. Previously, the system would have a
chance of booting in this situation. The manner in which kmod was
introduced makes this situation far more likely to cripple systems.
	If you want to understand the worst case scenario when dealing with
this regression, disable udev and reboot your system. You should have a
virtual terminal with no networking and no X. Should this happen with
systemd-udevd, then you would also have a hung systemd-udevd process
that you cannot kill. Attempts restart systemd-udevd should result in
more hung processes.

Where is development now?

	We have rewritten the build system and restored support for older
kernels and verified compatibility as far back as Linux 2.6.31. We have
tagged 1_beta1 and eudev is in the portage tree. A few lingering
dependency issues exist, but we should have them ironed out shortly.
	Work is on-going to introduce uclibc support, to reintroduce
70-persistent-net.rules and to reintroduce support for a separate /usr
support without an initramfs. Strictly speaking, we do not recommend
having a separate /usr mount, but we intend to support such
configurations in eudev indefinitely.
	We are also looking into fixing various regressions that the systemd
developers introduced, which include the kmod regressions.

What are your plans for future development?

	We intend to demonstrate to the current Gentoo udev maintainers that
maintaining systemd-udevd outside of the systemd package is unnecessary
when we have eudev. If they wish to continue maintaining systemd-udevd
as they are, then that is fine unless the Gentoo Council decides otherwise.
	After eudev has reached stable status in Gentoo, we intend to start
reaching out to other distributions to collaborate on further
development. Ideally, eudev will be something that all distributions can
use as a drop-in replacement for systemd-udevd.

Yours truly,
Richard Yao

P.S. I have BCCed a few people that might be interested in reading the
project announcement. Unfortunately, the gentoo-dev list requires
registration to avoid spammers, so you will need to register to reply to
the list. Documentation on how to register on the list is available at
the following address:

http://www.gentoo.org/main/en/lists.xml


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: [gentoo-project] eudev project announcement

[Rich Freeman]

On Fri, Dec 14, 2012 at 10:52 PM, Richard Yao <ryao@gentoo.org> wrote:
>         The systemd developers were in the middle of a transition to the LGPL
> from the GPL when we forked. We inherited the code in the middle of that
> transition and we see no reason to pursue a different course. Therefore,
> all future changes that we make to eudev will be available under the LGPL.

Not sure what the driver is to use LGPL, but in general the Gentoo
social contract requires that all contributions be made under GPLv2+
or the CC BY-SAv2+.  I'm sure exceptions can be made if they make
sense and are aligned with Gentoo's mission (likely something that
would fall on the Foundation to approve).  If eudev were a non-Gentoo
project then Gentoo could depend on it as long as it used any
OSI-approved license.

Why not just use GPLv2+?  The LGPL is compatible, so this would not
prevent us from merging udev changes.

Not suggesting that we shouldn't use the LGPL if there is a good
reason to do so that is aligned with Gentoo's mission.  I would just
like to understand the reason for it.

Rich

Re: [gentoo-project] eudev project announcement

[Richard Yao]

[-- Attachment #1: Type: text/plain, Size: 1951 bytes --]

On 12/15/2012 07:48 AM, Rich Freeman wrote:
> On Fri, Dec 14, 2012 at 10:52 PM, Richard Yao <ryao@gentoo.org> wrote:
>>         The systemd developers were in the middle of a transition to the LGPL
>> from the GPL when we forked. We inherited the code in the middle of that
>> transition and we see no reason to pursue a different course. Therefore,
>> all future changes that we make to eudev will be available under the LGPL.
> 
> Not sure what the driver is to use LGPL, but in general the Gentoo
> social contract requires that all contributions be made under GPLv2+
> or the CC BY-SAv2+.  I'm sure exceptions can be made if they make
> sense and are aligned with Gentoo's mission (likely something that
> would fall on the Foundation to approve).  If eudev were a non-Gentoo
> project then Gentoo could depend on it as long as it used any
> OSI-approved license.
> 
> Why not just use GPLv2+?  The LGPL is compatible, so this would not
> prevent us from merging udev changes.
> 
> Not suggesting that we shouldn't use the LGPL if there is a good
> reason to do so that is aligned with Gentoo's mission.  I would just
> like to understand the reason for it.
> 
> Rich
> 

The GPLv2 would be great if we wanted to prevent systemd and others from
being able to make use of our work. An argument could be made that we
should prevent them from being able to use it. However, our work is
meant for all, including people with different priorities. In addition,
Gentoo has a long history of respecting project licenses. Given that we
forked from a project that was transitioning to the LGPL, it would make
sense to continue on that course. If that is inappropriate, then we
would need some kind of Gentoo-wide decision to stop licensing patches
in the portage tree under non-GPL licenses. This would also prevent us
from patching software that is under GPL-incompatible, yet OSI-approved
licenses, such as the CDDL.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: [gentoo-project] eudev project announcement

[Samuli Suominen]

On 15/12/12 05:52, Richard Yao wrote:
> Dear Everyone,
>
> 	I am pleased to announce the Gentoo eudev project. Many of you already
> know about the eudev project from early publicity that we had before
> things were ready. Despite that, I hope to take advantage of the
> official announcement to explain what we are doing, why we are doing it
> and what it means for you. I have broken the announcement into
> subsections, each with a title for ease of reading.
>
> Why fork udev?
>
> 	Earlier this year, udev upstream was absorbed into systemd. udev often
> breaks compatibility with older systems by depending upon recent Linux
> kernel releases, even when such dependencies are avoidable. This became
> worse after udev became part of systemd, which has jeopardized our
> ability to support existing installations. The systemd developers are
> uninterested in providing full support in udev to systemd alternatives.
> These are problems for us and we have decided to fork udev to address them.
>
> You are a Gentoo project. What does this mean?
>
> 	Gentoo as an organization is quite similar to github, although it is
> exclusive to Gentoo developers. Our rules permit all Gentoo developers
> have the ability to start a project and such projects are entitled to be
> hosted on Gentoo infrastructure. This by no means constitutes official
> endorsement by Gentoo's governing body and we have no authority to
> dictate the future direction of Gentoo. We do have the ability to
> provide an alternative to Gentoo users, which we fully intend to do.
> 	eudev will be by no means exclusive to Gentoo. We will handle bug
> reports from users of other distribution in the same way that we handle
> bug reports involving Gentoo. This will be much like other Gentoo-hosted
> projects such as portage and OpenRC.
>
> What is your project's license?
>
> 	The systemd developers were in the middle of a transition to the LGPL
> from the GPL when we forked. We inherited the code in the middle of that
> transition and we see no reason to pursue a different course. Therefore,
> all future changes that we make to eudev will be available under the LGPL.
>
> What are your project's goals?
>
> 	Our primary goal is to address the problems with systemd-udev that
> caused us to fork it in the first place. In particular, we want better
> compatibility with existing software such as OpenRC and Upstart, older
> kernels, various toolchains and anything else required by users and
> various distributions. We also want to minimize regressions and work
> with developers of other distributions (and components used by them) to
> address issues.
>
> How will you minimize regressions?
>
> 	We intend to maintain HEAD in a releaseable state. All minor changes
> require review from one eudev developer and all major changes require
> review from two eudev developers. This does not include the author. In
> addition, we intend to require commits to make logically independent
> changes with descriptive commit messages to make regression hunting
> easier when regressions do happen.
> 	These rules were not enforced at the beginning, but we are
> transitioning toward enforcement. They will enter full effect once we
> tag our first release candidate.
>
> How do you intend to work with other distributions?
>
> 	We have our repository on github, which is known for easy
> collaboration. If a distribution developer identifies a problem with
> eudev, they can file an issue and we will do our best to resolve their
> problem. If they wish to work with us to resolve it, we can talk in IRC
> and they can also file pull requests. Provided that the changes are not
> entirely unreasonable (e.g. pushing an init system into udev), we are
> willing to work with authors of pull requests to get them into a
> mergeable state. The only hard rule is that changes cannot break the
> ability of existing systems to upgrade.
> 	We also plan to make an official mailing list, which will be hosted on
> Gentoo infrastructure.
>
> Will you make the boot process faster?
>
> 	We have ideas on how to make udevd faster. However, people usually only
> notice the time that udevd takes when there is a bug and we are more
> interested in fixing those bugs than we are in shaving milliseconds off
> boot time. There are plenty of areas that could use attention by people
> that are interested in a faster boot process before udev becomes one of
> them. We consider things such as a reliable boot process to be more
> important than speed and we are willing to subject users to the
> additional few hundred milliseconds that failing to tweak things for
> speed incurs.
> 	We are already dealing with regressions that the systemd developers
> introduced in their attempt to make things faster and we consider fixing
> them to be a priority. However, we would be happy to collaborate with
> people willing to work on boot speed improvements to get them into a
> mergeable state. We encourage people interested in speed improvements to
> talk to us about how they could be done in a reasonable manner. It would
> be premature to do it at this instant, but it would definitely be
> something that could be done after we have tagged a few stable releases.
>
> What did systemd break when trying to make things faster?
>
> 	A good example involves module loading. Previously, each module load
> would incur a roughly 10 to 20 ms disk access latency and a 0.01 ms fork
> overhead. This was partially masked by udev's ability to execute rules
> asynchronously through fork(), which meant that multiple modules could
> be read in parallel.
> 	The introduction of kmod eliminated the 0.01 ms fork() overhead, and
> consequently required each module to be loaded sequentially. This
> imposes an overhead of 10 to 20 milliseconds times the number of
> modules, which is asymptotically worse than what it replaced. A feature
> of kmod intended to address that placed all modules into a single file,
> which would actually make things faster. However, none of our users use
> it and all of our users would suffer from it.
> 	In addition, the manner in which kmod was integrated has implications
> beyond speed regressions. The use of kmod by udev permits a buggy kernel
> module (possibly interacting with bad hardware) to hang in module_init.
> This causes udevd to hang, which prevents further rule processing. This
> is a bad situation, but we feel that it is important to handle bad
> situations in a graceful manner. Previously, the system would have a
> chance of booting in this situation. The manner in which kmod was
> introduced makes this situation far more likely to cripple systems.
> 	If you want to understand the worst case scenario when dealing with
> this regression, disable udev and reboot your system. You should have a
> virtual terminal with no networking and no X. Should this happen with
> systemd-udevd, then you would also have a hung systemd-udevd process
> that you cannot kill. Attempts restart systemd-udevd should result in
> more hung processes.
>
> Where is development now?
>
> 	We have rewritten the build system and restored support for older
> kernels and verified compatibility as far back as Linux 2.6.31. We have
> tagged 1_beta1 and eudev is in the portage tree. A few lingering
> dependency issues exist, but we should have them ironed out shortly.
> 	Work is on-going to introduce uclibc support, to reintroduce
> 70-persistent-net.rules and to reintroduce support for a separate /usr
> support without an initramfs. Strictly speaking, we do not recommend
> having a separate /usr mount, but we intend to support such
> configurations in eudev indefinitely.
> 	We are also looking into fixing various regressions that the systemd
> developers introduced, which include the kmod regressions.
>
> What are your plans for future development?
>
> 	We intend to demonstrate to the current Gentoo udev maintainers that
> maintaining systemd-udevd outside of the systemd package is unnecessary
> when we have eudev. If they wish to continue maintaining systemd-udevd
> as they are, then that is fine unless the Gentoo Council decides otherwise.
> 	After eudev has reached stable status in Gentoo, we intend to start
> reaching out to other distributions to collaborate on further
> development. Ideally, eudev will be something that all distributions can
> use as a drop-in replacement for systemd-udevd.
>
> Yours truly,
> Richard Yao
>
> P.S. I have BCCed a few people that might be interested in reading the
> project announcement. Unfortunately, the gentoo-dev list requires
> registration to avoid spammers, so you will need to register to reply to
> the list. Documentation on how to register on the list is available at
> the following address:
>
> http://www.gentoo.org/main/en/lists.xml
>

Despite what you may have thought or seen my opinions to be earlier, I 
wish best of luck to you.

I don't have plans on trying eudev, and trust you will continue to 
backport relevent code from orig. upstream git so people don't have to 
cross-post to 2 different places. I also hope eudev will never require 
code to other packages, in a way of keeping the compability with orig. 
upstream to keep things maintainable as they are now.

Thanks,
Samuli