[gentoo-dev] Stability of /sys api

[gentoo-dev] Stability of /sys api

[Walter Dnes]

  After some Google-searching, I think I've figured out how to implement
automounting under mdev.  I'd like to put in as much sanity-checking
into the script as possible.  Right now I have 1 USB stick plugged in as
/dev/sdb.  Th hard drive is /dev/sda.  The "removable" data is readable
like so...

cat /sys/block/sda/removable
0

cat /sys/block/sdb/removable 
1

  My question... is this API stable or deprecated?  I.e. can I count on
it being around for a while?  I figure this question is a developer type
question rather than ordinary user type.

-- 
Walter Dnes <waltdnes@waltdnes.org>

Re: [gentoo-dev] Stability of /sys api

[Brian Harring]

On Mon, May 14, 2012 at 03:53:53AM -0400, Walter Dnes wrote:
>   After some Google-searching, I think I've figured out how to implement
> automounting under mdev.  I'd like to put in as much sanity-checking
> into the script as possible.  Right now I have 1 USB stick plugged in as
> /dev/sdb.  Th hard drive is /dev/sda.  The "removable" data is readable
> like so...
> 
> cat /sys/block/sda/removable
> 0
> 
> cat /sys/block/sdb/removable 
> 1
> 
>   My question... is this API stable or deprecated?  I.e. can I count on
> it being around for a while?  I figure this question is a developer type
> question rather than ordinary user type.

Api is stable although last I dealt with that crap it was reliant on 
chipsets/controllers not sucking and misreporting (mmc in particular 
comes to mind, although perhaps the hardware sucks less these days).

Suggest you start studying udev source in addition since your 
questions of that sort are likely to be answered there.  Aka, most 
likely wind up asking udev upstream (likely gregkh assuming he hasn't 
killfile'd everyone from that thread).

Now the unfun part; this isn't really the right place to be asking.  I 
get you're doing this w/ a gentoo intent, but you're dancing that line 
mightily fine.  People asking "can I safely use nested context 
managers in python2.6" even if it's orientated towards a potential 
gentoo bit (say the ephemeral gentoo stats project), typically will be 
told "dunno.  Ask the people who wrote the damn thing".

Point there is that the ml shouldn't be used as tech help for the guts 
of "I don't want udev and am trying to replace it with mdev"; 
pkgcore nor paludis internal questions don't come here (format does 
since this is the appropriate venue) under the same logic.  Forums 
come to mind, or appropriate upstreams as mentioned.

Barring that, use the source luke, and start reading the lkml.  If 
you're trying to do this, you'll likely need to track discussions 
there.

Not trying to be a dick mind you, and perhaps others view othewise, 
but this isn't the place for it nor do I suspect people care to see 
more of this particular war play out on our ml.

~harring

Re: [gentoo-dev] Stability of /sys api

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 317 bytes --]

On Monday 14 May 2012 03:53:53 Walter Dnes wrote:
>   My question... is this API stable or deprecated?  I.e. can I count on
> it being around for a while?  I figure this question is a developer type
> question rather than ordinary user type.

if userspace is relying on stuff in /sys, then it's part of the ABI
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Stability of /sys api

[Greg KH]

On Mon, May 14, 2012 at 12:09:23PM -0400, Mike Frysinger wrote:
> On Monday 14 May 2012 03:53:53 Walter Dnes wrote:
> >   My question... is this API stable or deprecated?  I.e. can I count on
> > it being around for a while?  I figure this question is a developer type
> > question rather than ordinary user type.
> 
> if userspace is relying on stuff in /sys, then it's part of the ABI

Yes, but note, you are looking at the wrong thing, what you are reading
isn't really what you think it means...

And to rely on sysfs, you have to be very careful, you can not rely on
position, or files, always being where you saw them yesterday as device
ids, topologies, and other stuff, change all the time.

greg k-h

Re: [gentoo-dev] Stability of /sys api

[James Cloos]

WD> cat /sys/block/sda/removable
WD> 0

Note that a 0 there does not imply that the device cannot hotplug.

My USB drive reports 0.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [gentoo-dev] Stability of /sys api

[Olivier Crête]

[-- Attachment #1: Type: text/plain, Size: 381 bytes --]

On Mon, 2012-05-14 at 12:31 -0400, James Cloos wrote:
> WD> cat /sys/block/sda/removable
> WD> 0
> 
> Note that a 0 there does not imply that the device cannot hotplug.
> 
> My USB drive reports 0.

And I'm sure it works fine with udev?

"Those who do not understand udev are condemned to reinvent it, poorly".

-- 
Olivier Crête
tester@gentoo.org
Gentoo Developer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Stability of /sys api

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 427 bytes --]

On Mon, 14 May 2012 12:56:39 -0400
Olivier Crête <tester@gentoo.org> wrote:
> On Mon, 2012-05-14 at 12:31 -0400, James Cloos wrote:
> > WD> cat /sys/block/sda/removable
> > WD> 0
> > 
> > Note that a 0 there does not imply that the device cannot hotplug.
> > 
> > My USB drive reports 0.
> 
> And I'm sure it works fine with udev?

I dunno. Internet Explorer broke and now udev won't run.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Stability of /sys api

[James Cloos]

>>>>> "OC" == Olivier Crête <tester@gentoo.org> writes:

OC> And I'm sure it works fine with udev?

It automounts when plugged in, if that is what you mean.  (In fact each
partition does; the one in fstab(5) where it should and the one not in
fstab in a mount point based on its label.)

And the dev files get removed when the drive is unplugged.

(unplugged here means either the usb cable or the power cable.)

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [gentoo-dev] Stability of /sys api

[Walter Dnes]

On Mon, May 14, 2012 at 12:31:25PM -0400, James Cloos wrote
> WD> cat /sys/block/sda/removable
> WD> 0
> 
> Note that a 0 there does not imply that the device cannot hotplug.
> 
> My USB drive reports 0.

  You're right.  Same for me.  Thanks for pointing it out.

-- 
Walter Dnes <waltdnes@waltdnes.org>

Re: [gentoo-dev] Stability of /sys api

[James Cloos]

>> My USB drive reports 0.

WD>   You're right.  Same for me.  Thanks for pointing it out.

The removable flag specifies whether the drive has removable media;
before flash drives only things like floppy, optical, zip, etc drives
had removable==1.  It also would be accurate for flash card readers.
If thumb drives specify it, then they lie.

The flag is read from the drive's metadata.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [gentoo-dev] Stability of /sys api

[Greg KH]

On Mon, May 14, 2012 at 03:53:53AM -0400, Walter Dnes wrote:
>   After some Google-searching, I think I've figured out how to implement
> automounting under mdev.  I'd like to put in as much sanity-checking
> into the script as possible.  Right now I have 1 USB stick plugged in as
> /dev/sdb.  Th hard drive is /dev/sda.  The "removable" data is readable
> like so...
> 
> cat /sys/block/sda/removable
> 0
> 
> cat /sys/block/sdb/removable 
> 1
> 
>   My question... is this API stable or deprecated?  I.e. can I count on
> it being around for a while?  I figure this question is a developer type
> question rather than ordinary user type.

You might want to look at Documentation/ABI/ in the kernel source tree
if you are curious about sysfs files like this.

As you have figured out, this really doesn't mean what you think it
does.  In fact, there's no way to determine what I think you are asking
for, which is "can this disk go away", as really, any disk can go away
at any point in time, just like any disk can show up at any point in
time as well.  Think PCI hotplug systems, thunderbolt, pcmcia,
ExpressCard, Firewire, USB, SCSI, etc.

So you need to implement stuff such that you are not dependant on the
bus type.  If you see a new disk, act on it, it's that simple.

But note, please do not be automounting disks from uevents directly.  As
someone else said on this thread, "those that have not learned from the
lessons of udev, will implement it poorly".  We learned that this is not
a good idea at all, and should be left to userspace helper applications
that listen for dbus messages.  Both GNOME and KDE work this way quite
well, so I would be very wary of reimplementing the wheel.

Actually with all the hype about mdev these days, why not just use a 3
year old version of udev (or maybe 4), that is probably what mdev is at
as far as functionality goes.  Why not just fork udev from then and go
forward from that?  What exactly are you not liking in udev that makes
you want to get rid of it so badly?  What is it doing that bothers
people so much?

Good luck,

greg k-h

Re: [gentoo-dev] Stability of /sys api

[Maxim Kammerer]

On Tue, May 15, 2012 at 4:23 AM, Greg KH <gregkh@gentoo.org> wrote:
> We learned that this is not a good idea at all, and should be left to userspace helper applications
> that listen for dbus messages.

Could you perhaps expand a bit on those reasons? E.g., I had good
experience with the following short script for coupling udev events
with autofs: https://github.com/mkdesu/liberte/blob/master/src/usr/local/sbin/ps-mount.
Gentoo wiki has a similar tutorial as well. Granted, it is a
single-user setup, but I can imagine it being extended to work with
ConsoleKit. One obvious problem is mounting encrypted volumes. I
thought about moving to e.g., udisks-glue (as a more standard
solution), but from what I hear there are too many bugs with udisks at
the moment.

> Actually with all the hype about mdev these days, why not just use a 3
> year old version of udev (or maybe 4), that is probably what mdev is at
> as far as functionality goes.

I don't know at what state udev was 3 or 4 years ago, but mdev can:

1. Populate /dev (now unnecessary due to devtmpfs).
2. Handle ownership, permissions and symlinks to /dev nodes once they
appear, according to simple rules (can be probably done with inotify).
3. Act as /sbin/hotplug, typically doing something equivalent to this one-liner:
   [ "${ACTION}" = add  -a  -n "${MODALIAS}" ] && modprobe -qb "${MODALIAS}"

I don't think mdev can do anything else. Building any serious
framework on top of mdev seems pointless to me, since it will probably
end up as a small subset of udev core reimplemented with scripts.

-- 
Maxim Kammerer
Liberté Linux (discussion / support: http://dee.su/liberte-contribute)

Re: [gentoo-dev] Stability of /sys api

[Walter Dnes]

On Tue, May 15, 2012 at 04:56:15AM +0300, Maxim Kammerer wrote

> I don't know at what state udev was 3 or 4 years ago, but mdev can:
> 
> 1. Populate /dev (now unnecessary due to devtmpfs).
> 2. Handle ownership, permissions and symlinks to /dev nodes once they
> appear, according to simple rules (can be probably done with inotify).
> 3. Act as /sbin/hotplug, typically doing something equivalent to this one-liner:
>    [ "${ACTION}" = add  -a  -n "${MODALIAS}" ] && modprobe -qb "${MODALIAS}"

  That's *EXACTLY* what I want and need.  To borrow an old emacs joke,
udev is a mediocre OS that lacks a lightweight device manager.

> I don't think mdev can do anything else. Building any serious
> framework on top of mdev seems pointless to me, since it will probably
> end up as a small subset of udev core reimplemented with scripts.

  I *DON'T WANT* "a serious framework", I want a lightweight device
manager... period... end of story.  Stick with the unix principle of one
app doing one thing well.  mdev is enough for the vast majority of people.

-- 
Walter Dnes <waltdnes@waltdnes.org>

Re: [gentoo-dev] Stability of /sys api

[Olivier Crête]

[-- Attachment #1: Type: text/plain, Size: 534 bytes --]

On Tue, 2012-05-15 at 01:05 -0400, Walter Dnes wrote:
>   I *DON'T WANT* "a serious framework", I want a lightweight device
> manager... period... end of story.  Stick with the unix principle of one
> app doing one thing well.  mdev is enough for the vast majority of people.

For the people who don't want to easily use USB sticks or digital
cameras or gsm dongles or really any modern hardware, I'm sure mdev is
fine. A static /dev is even fine for you probably.


-- 
Olivier Crête
tester@gentoo.org
Gentoo Developer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Stability of /sys api

[Canek Peláez Valdés]

On Tue, May 15, 2012 at 1:32 AM, Olivier Crête <tester@gentoo.org> wrote:
> On Tue, 2012-05-15 at 01:05 -0400, Walter Dnes wrote:
>>   I *DON'T WANT* "a serious framework", I want a lightweight device
>> manager... period... end of story.  Stick with the unix principle of one
>> app doing one thing well.  mdev is enough for the vast majority of people.
>
> For the people who don't want to easily use USB sticks or digital
> cameras or gsm dongles or really any modern hardware, I'm sure mdev is
> fine. A static /dev is even fine for you probably.

I agree. And I don't believe people "who don't want to easily use USB
sticks or digital cameras or gsm dongles or really any modern
hardware" qualify as "the vast majority of people".

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México

Re: [gentoo-dev] Stability of /sys api

[Walter Dnes]

On Tue, May 15, 2012 at 02:32:57AM -0400, Olivier Cr?te wrote
> On Tue, 2012-05-15 at 01:05 -0400, Walter Dnes wrote:
> >   I *DON'T WANT* "a serious framework", I want a lightweight device
> > manager... period... end of story.  Stick with the unix principle of one
> > app doing one thing well.  mdev is enough for the vast majority of people.
> 
> For the people who don't want to easily use USB sticks or digital
> cameras or gsm dongles or really any modern hardware, I'm sure mdev is
> fine. A static /dev is even fine for you probably.

  Huh!?!?!?  USB sticks work just fine, thank you, with mdev.  I also
regularly backup my mdev-based machine via rsync to an external drive
via USB.  And yes my camera does show up as a USB mass storage device.
Ditto for my HTC Desire.

-- 
Walter Dnes <waltdnes@waltdnes.org>

Re: [gentoo-dev] Stability of /sys api

[Nirbheek Chauhan]

On Wed, May 16, 2012 at 1:59 AM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> On Tue, May 15, 2012 at 02:32:57AM -0400, Olivier Cr?te wrote
>> On Tue, 2012-05-15 at 01:05 -0400, Walter Dnes wrote:
>> >   I *DON'T WANT* "a serious framework", I want a lightweight device
>> > manager... period... end of story.  Stick with the unix principle of one
>> > app doing one thing well.  mdev is enough for the vast majority of people.
>>
>> For the people who don't want to easily use USB sticks or digital
>> cameras or gsm dongles or really any modern hardware, I'm sure mdev is
>> fine. A static /dev is even fine for you probably.
>
>  Huh!?!?!?  USB sticks work just fine, thank you, with mdev.  I also
> regularly backup my mdev-based machine via rsync to an external drive
> via USB.  And yes my camera does show up as a USB mass storage device.
> Ditto for my HTC Desire.
>

"Those who don't understand UNI^H^H^Hsoftware are condemned to
reinvent it, poorly."

-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] Stability of /sys api

[Greg KH]

On Tue, May 15, 2012 at 01:05:57AM -0400, Walter Dnes wrote:
> On Tue, May 15, 2012 at 04:56:15AM +0300, Maxim Kammerer wrote
> 
> > I don't know at what state udev was 3 or 4 years ago, but mdev can:
> > 
> > 1. Populate /dev (now unnecessary due to devtmpfs).
> > 2. Handle ownership, permissions and symlinks to /dev nodes once they
> > appear, according to simple rules (can be probably done with inotify).
> > 3. Act as /sbin/hotplug, typically doing something equivalent to this one-liner:
> >    [ "${ACTION}" = add  -a  -n "${MODALIAS}" ] && modprobe -qb "${MODALIAS}"
> 
>   That's *EXACTLY* what I want and need.  To borrow an old emacs joke,
> udev is a mediocre OS that lacks a lightweight device manager.

Huh?  How is udev not "lightweight"?  What does it have in it that makes
it "heavy"?  I see lots of things in mdev that make it heavier and
slower than udev :)

> > I don't think mdev can do anything else. Building any serious
> > framework on top of mdev seems pointless to me, since it will probably
> > end up as a small subset of udev core reimplemented with scripts.
> 
>   I *DON'T WANT* "a serious framework", I want a lightweight device
> manager... period... end of story.  Stick with the unix principle of one
> app doing one thing well.  mdev is enough for the vast majority of people.

I don't see how udev isn't a "do one thing really well" program and pass
off to others, piping data to programs that can do other things to it if
wanted/needed.  Can you explain how it violates this Unix maxium?

thanks,

greg k-h

Re: [gentoo-dev] Stability of /sys api

[Greg KH]

On Tue, May 15, 2012 at 04:56:15AM +0300, Maxim Kammerer wrote:
> On Tue, May 15, 2012 at 4:23 AM, Greg KH <gregkh@gentoo.org> wrote:
> > We learned that this is not a good idea at all, and should be left to userspace helper applications
> > that listen for dbus messages.
> 
> Could you perhaps expand a bit on those reasons? E.g., I had good
> experience with the following short script for coupling udev events
> with autofs: https://github.com/mkdesu/liberte/blob/master/src/usr/local/sbin/ps-mount.
> Gentoo wiki has a similar tutorial as well. Granted, it is a
> single-user setup, but I can imagine it being extended to work with
> ConsoleKit. One obvious problem is mounting encrypted volumes. I
> thought about moving to e.g., udisks-glue (as a more standard
> solution), but from what I hear there are too many bugs with udisks at
> the moment.

I know of no such problem with udisks, have you reported them to the
upstream developers?

Yes, encrypted disks are one such problem, other bad things happen when
you hit a disk with bad sectors, and other "fun" things.  You can stall
the whole hotplug path, causing issues and overruns.

For more details as to why this is a bad idea, see the linux-hotplug
mailing list archives, it was covered in detail there a few years ago.

> > Actually with all the hype about mdev these days, why not just use a 3
> > year old version of udev (or maybe 4), that is probably what mdev is at
> > as far as functionality goes.
> 
> I don't know at what state udev was 3 or 4 years ago, but mdev can:
> 
> 1. Populate /dev (now unnecessary due to devtmpfs).

udev can't even do that these days (see, udev got smaller), as devtmpfs
is what needs to do this.

Right there, if that's all you need, and it's what most embedded systems
need, udev isn't even needed, just use devtmpfs and all is fine.

> 2. Handle ownership, permissions and symlinks to /dev nodes once they
> appear, according to simple rules (can be probably done with inotify).

Careful about user ownerships of multi-user machines, that gets tricky.

> 3. Act as /sbin/hotplug, typically doing something equivalent to this one-liner:
>    [ "${ACTION}" = add  -a  -n "${MODALIAS}" ] && modprobe -qb "${MODALIAS}"

Oh wow.  Your /proc/sys/kernel/hotplug file should be set to "" if you
want any chance to have a fast boot process.

So, using mdev you slow your boot down, and slow down the response time
from uevents.  I don't think that's what you really want for a
"lightweight" system :)

> I don't think mdev can do anything else. Building any serious
> framework on top of mdev seems pointless to me, since it will probably
> end up as a small subset of udev core reimplemented with scripts.

Again, what's wrong with just using udev for this as-is?  What benifit
can mdev provide you over udev?  Is it somehow smaller?  How small?
It's obviously not faster due to the forking required from
/sbin/hotplug, so that can't be a good reason to prefer it.

thanks,

greg k-h

Re: [gentoo-dev] Stability of /sys api

[Maxim Kammerer]

On Tue, May 15, 2012 at 9:21 PM, Greg KH <gregkh@gentoo.org> wrote:
> I know of no such problem with udisks, have you reported them to the
> upstream developers?

As I said, it's just what I hear — perhaps it's the usual retrograde
whining. I should probably just try udisks-glue, the only issue I see
is that it depends on UDisks 1.

> You can stall the whole hotplug path, causing issues and overruns.

Note that the script I mentioned immediately forks, and only updates
autofs mapping entries — it doesn't actually mount anything.

> Right there, if that's all you need, and it's what most embedded systems
> need, udev isn't even needed, just use devtmpfs and all is fine.

Perhaps I wasn't expressing myself clearly — the point was to show
that mdev is not needed. I used mdev in initramfs previously, but now
rely on devtmpfs and the one-liner /sbin/hotplug for loading modules.
Incidentally, turning off mdev resulted in Busybox executable size
change of precisely 0, which I guess shows how much mdev actually
does.

> I really want to get rid of that entry and option from the kernel entirely

Please don't — it's useful in Busybox-based initramfs. Dracut-like
inclusion of udev and its dependencies DAG is usually unnecessary and
an overkill.

-- 
Maxim Kammerer
Liberté Linux (discussion / support: http://dee.su/liberte-contribute)

Re: [gentoo-dev] Stability of /sys api

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 790 bytes --]

On Mon, May 14, 2012 at 06:23:36PM -0700, Greg KH wrote:
> Actually with all the hype about mdev these days, why not just use a 3
> year old version of udev (or maybe 4), that is probably what mdev is at
> as far as functionality goes.  Why not just fork udev from then and go
> forward from that?  What exactly are you not liking in udev that makes
> you want to get rid of it so badly?  What is it doing that bothers
> people so much?

I'm wondering the same thing since once busybox 1.20.0 hits stable you
will be able to have a separate /usr without an initramfs quite easily
if that's what you want to do.

When you emerge this version of busybox with the "sep-usr" use flag, you
get a binary in / called ginit. Now just follow the instructions you got
when you emerged it.

William



[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: Stability of /sys api

[Steven J Long]

William Hubbs wrote:
> I'm wondering the same thing since once busybox 1.20.0 hits stable you
> will be able to have a separate /usr without an initramfs quite easily
> if that's what you want to do.
> 
> When you emerge this version of busybox with the "sep-usr" use flag, you
> get a binary in / called ginit. Now just follow the instructions you got
> when you emerged it.
> 
Thing is it runs before the real init[1] so if we are using a separate /usr 
partition on LVM, will it still work? I'd have thought not, since we need 
the device-mapper service and there's /etc/lvm.conf to consider, but I'll 
gladly be told different.

(/usr on lvm was the example given in the Council discussion, and has been 
mentioned in docs, but I'd imagine same applies to /usr on mdadm, if not 
auto-detected.)

[1] http://www.gossamer-threads.com/lists/gentoo/dev/252734
-- 
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

Re: [gentoo-dev] Re: Stability of /sys api

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 730 bytes --]

Hi Steven,

On Thu, May 17, 2012 at 06:48:33AM +0100, Steven J Long wrote:
> Thing is it runs before the real init[1] so if we are using a separate /usr 
> partition on LVM, will it still work? I'd have thought not, since we need 
> the device-mapper service and there's /etc/lvm.conf to consider, but I'll 
> gladly be told different.
 
No, you are correct about this. This does not work if you have /usr on
lvm, mdadm, or encrypted. The same applies to /. That is the situation
where you would need an initramfs.

I'm curious, have you seen our initramfs guide yet [1]? Making and using
an initramfs seems to be pretty well documented these days.

William

[1] http://www.gentoo.org/doc/en/initramfs-guide.xml

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Implementing udev without an initramfs (Was: Stability of /sys api)

[Steven J Long]

Hey, William

William Hubbs wrote:
> Steven J Long wrote:
>> Thing is it runs before the real init[1] so if we are using a separate
>> /usr partition on LVM, will it still work? I'd have thought not, since we
>> need the device-mapper service and there's /etc/lvm.conf to consider, but
>> I'll gladly be told different.
>  
> No, you are correct about this. This does not work if you have /usr on
> lvm, mdadm, or encrypted. The same applies to /. That is the situation
> where you would need an initramfs.
>
Let's not conflate the two: we never needed an initramfs before now*, unless 
/ were on lvm or mdadm (not sure about dmcrypt, but always thought it 
pointless without encrypting root.)

So the touted solution to Chainsaw's problem, as raised to Council, doesn't 
actually work for that use-case, and there is still no official support for 
both setups.

--
* If anyone wants to chip in with something along the lines of "you did but 
you didn't know it" or "this isn't safe" etc, please don't: I've heard it 
all before, and acknowledged technical points, and you'll just be adding 
noise and distracting from the topic. The users we're talking about have 
already set this up manually, if that's your concern.
--
> I'm curious, have you seen our initramfs guide yet [1]? Making and using
> an initramfs seems to be pretty well documented these days.
>
No, that is useful information to have, though all the detail is in links 
I've seen before. Also useful are:

NeddySeagoon's Rough Guide to udev-182+:
	http://forums.gentoo.org/viewtopic-t-920644.html
udeved's port of Arch's mkinitcpio:
	http://forums.gentoo.org/viewtopic-t-923168.html
..which seems like a really nice tool to build an initramfs.

But let's be frank: you're never going to persuade me, or many others[2], 
that merging udev and systemd, as well as / and /usr, and running an 
initramfs is the One True Way(TM). And the argument is boring, for all of us 
I'm sure, as the title of [2] should make clear.

So, let's agree that no-one needs to fork udev at this stage. There /are/ 
methods that work flawlessly[3] with lvm and separate /usr with no 
initramfs, at least for now, should people care to explore them[4], and give 
feedback to improve them.

What could spoil that and force a fork (or a switch to a patched mdev) 
instead of initscript-ordering?

1) Random linkage into /usr/lib

  It's simple enough to check linkage, and while it would be nice if there 
were a portage feature to check any binaries installed to /bin /sbin or 
/lib* and ewarn if they link outside /lib* (QA otherwise), we can happily 
implement a portage bashrc hook to do it for us. Since every package manager 
uses a staged build and install, it doesn't matter what the end-user's 
filesystem layout is; even if they've munged all binaries into /usr, the 
image won't use symlinks, but contain directories.

  I accept that this is something we're going to have to handle on our own, 
since initramfs-people aren't interested in the data, as their scripts 
already automate pulling in dependent libs.

  But atm this is a non-issue. Running this one-liner:
for f in /bin/* /sbin/*; do ldd "$f"|grep -q /usr || continue; echo "** $f";
ldd "$f"; done
..shows only /sbin/umount.udisks, an optional desktop dep, linking to /usr 
here.

 Ultimately, we may need a tiny overlay for the few packages that are both 
A) required in early boot before localmount, in the opinion of someone who 
wants to submit an ebuild or patches to it, and:
b) whose root-installed binaries link outside, or:
c) it uses: econf --prefix=/usr    and we'd like:
	  econf --prefix=/ --exec-prefix=/usr
and:
D) where upstream is unresponsive, and:
E) the ebuild maintainer doesn't want to accept a patch

  In most cases, we should be able to deal with (b) with portage hooks which 
move libs in "$D". I wouldn't like to do that automatically when a warning 
is flagged, though, as it should be specifically edited-in.

2) Hard dependency on systemd

  This is the killer, and would force a fork, or we'd have to start work on 
extensions to mdev. Outside our control, and pointless to speculate on, 
though given recent feedback, the old kernel.org udev repo[5] is a reference 
point for anyone who wants to check it out. I'd personally be wary of any 
changes to udev in systemd's repo[6] going forward, although it would only 
be an issue if and when unstable udev stopped working for openrc users.

Regards,
Steve.
 
> [1] http://www.gentoo.org/doc/en/initramfs-guide.xml
[2] http://forums.gentoo.org/viewtopic-t-921140.html
[3] http://forums.gentoo.org/viewtopic-t-901206.html
[4] http://forums.gentoo.org/viewtopic-t-918466.html
[5] http://git.kernel.org/?p=linux/hotplug/udev.git;a=summary
-- 
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

Re: [gentoo-dev] Implementing udev without an initramfs (Was: Stability of /sys api)

[Samuli Suominen]

On 05/19/2012 10:36 PM, Steven J Long wrote:
> 1) Random linkage into /usr/lib
> ..shows only /sbin/umount.udisks, an optional desktop dep, linking to /usr
> here.

I know you propably meant this as an example, but I've fixed it months 
ago by moving it to /usr:

http://bugs.gentoo.org/398081

Then vapier kindly fixed util-linux's umount to look for umount.* 
wrappers also from /usr:

http://bugs.gentoo.org/403073

I'm only waiting for new util-linux to stabilize, to stabilize new 
udisks, to get it out of /

(And rest of your post was boring repeat of the old. If you don't have 
anything new and constructive to add, I recommend staying quiet or at 
least moving this to some other ML, like gentoo-user.)

- Samuli

Re: [gentoo-dev] Implementing udev without an initramfs (Was: Stability of /sys api)

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

On Sat, 19 May 2012 20:36:57 +0100
Steven J Long <slong@rathaus.eclipse.co.uk> wrote:

> * If anyone wants to chip in with something along the lines of "you
> did but you didn't know it" or "this isn't safe" etc, please don't:
> I've heard it all before, and acknowledged technical points, and
> you'll just be adding noise and distracting from the topic.

* If anyone wants to chip in with something along the lines 'I never
needed an initramfs and nothing can change', please don't: we've heard
that a thousand times, and it gets boring.

In fact, if you just set up an initramfs instead of writing that post,
you'd probably save some time. Our time, especially.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Stability of /sys api

[Walter Dnes]

On Mon, May 14, 2012 at 06:23:36PM -0700, Greg KH wrote

> So you need to implement stuff such that you are not dependant on the
> bus type.  If you see a new disk, act on it, it's that simple.
> 
> But note, please do not be automounting disks from uevents directly.

  After some more Google-searching. it looks like the "official
channels" way is via /etc/mdev.conf.  Note that this is on a system with
busybox[mdev] and no udev.  /etc/mdev.conf has a rudimentary set of
"mdev rules" abilities, and most importantly, it can also call external
executables (scripts/programs/whatever).  On my mdev based machines...

$ cat /proc/sys/kernel/hotplug
/sbin/mdev

> Actually with all the hype about mdev these days, why not just use a 3
> year old version of udev (or maybe 4), that is probably what mdev is at
> as far as functionality goes.  Why not just fork udev from then and go
> forward from that?  What exactly are you not liking in udev that makes
> you want to get rid of it so badly?  What is it doing that bothers
> people so much?

  Unfortunately, I am not a C programmer, so forking udev is only a
dream.  As Maxim has pointed out, mdev does what most people need.  The
busybox people do the maintenance.  Given their target audience
(embedded and lightweight systems), we can be certain that mdev won't
grow into a monstrosity.  Even if I could do it, why reinvent the wheel?
We have a perfectly usable alternative right now in mdev.

  My main programming strength is bash scripts etc.  Actually, a "fork"
I'd be interested in would be to take standard Gentoo and replace as
many programs as possible with their busybox-symlink equivalants.  This
would require at least a new profile, to
a) create the appropriate symlinks, and
b) not pull in the standalone versions.

-- 
Walter Dnes <waltdnes@waltdnes.org>

Re: [gentoo-dev] Stability of /sys api

[Greg KH]

On Tue, May 15, 2012 at 01:55:23AM -0400, Walter Dnes wrote:
> On Mon, May 14, 2012 at 06:23:36PM -0700, Greg KH wrote
> 
> > So you need to implement stuff such that you are not dependant on the
> > bus type.  If you see a new disk, act on it, it's that simple.
> > 
> > But note, please do not be automounting disks from uevents directly.
> 
>   After some more Google-searching. it looks like the "official
> channels" way is via /etc/mdev.conf.  Note that this is on a system with
> busybox[mdev] and no udev.  /etc/mdev.conf has a rudimentary set of
> "mdev rules" abilities, and most importantly, it can also call external
> executables (scripts/programs/whatever).  On my mdev based machines...
> 
> $ cat /proc/sys/kernel/hotplug
> /sbin/mdev

Oh my, don't do that.  Please.  It will slow your machine down, and on
some systems, and devices, cause fork-bombs causing your box to come to
a crawl and potentially die.  There's also ordering issues that the
called program needs to resolve to get things right that add lots of
logic and slowdowns at times.  I really want to get rid of that entry
and option from the kernel entirely, but need to keep it due to legacy
systems and API issues.

But really, don't do that, it's not a good idea at all.

> > Actually with all the hype about mdev these days, why not just use a 3
> > year old version of udev (or maybe 4), that is probably what mdev is at
> > as far as functionality goes.  Why not just fork udev from then and go
> > forward from that?  What exactly are you not liking in udev that makes
> > you want to get rid of it so badly?  What is it doing that bothers
> > people so much?
> 
>   Unfortunately, I am not a C programmer, so forking udev is only a
> dream.  As Maxim has pointed out, mdev does what most people need.

As does udev.  And udev is faster, and I'd bet, the same size.

> The busybox people do the maintenance.  Given their target audience
> (embedded and lightweight systems), we can be certain that mdev won't
> grow into a monstrosity.  Even if I could do it, why reinvent the
> wheel?  We have a perfectly usable alternative right now in mdev.

mdev is the reinvention, but hey, if you want it, that's fine.  Embedded
and "lightweight" systems don't even need mdev or udev, they just use
the kernel-provided devtmpfs and all is good.

But it's your choice, and that's great, but please be aware of the
tradeoffs you are choosing, and again, my worry about using a
/sbin/hotplug executable is real and should be resolved.

What specifically is your objection to udev today?  Is it doing things
you don't like?  Too big?  Something else?

thanks,

greg k-h

Re: [gentoo-dev] Stability of /sys api

[Walter Dnes]

On Tue, May 15, 2012 at 11:26:03AM -0700, Greg KH wrote
> On Tue, May 15, 2012 at 01:55:23AM -0400, Walter Dnes wrote:
> > 
> >   After some more Google-searching. it looks like the "official
> > channels" way is via /etc/mdev.conf.  Note that this is on a system with
> > busybox[mdev] and no udev.  /etc/mdev.conf has a rudimentary set of
> > "mdev rules" abilities, and most importantly, it can also call external
> > executables (scripts/programs/whatever).  On my mdev based machines...
> > 
> > $ cat /proc/sys/kernel/hotplug
> > /sbin/mdev
> 
> Oh my, don't do that.  Please.  It will slow your machine down, and on
> some systems, and devices, cause fork-bombs causing your box to come to
> a crawl and potentially die.  There's also ordering issues that the
> called program needs to resolve to get things right that add lots of
> logic and slowdowns at times.  I really want to get rid of that entry
> and option from the kernel entirely, but need to keep it due to legacy
> systems and API issues.
> 
> But really, don't do that, it's not a good idea at all.

  During bootup, mdev is invoked as "mdev -s", and in hotplug mode it's
simply "mdev".  In hotplug mode, mdev returns some environmental
variables, specifically "ACTION", which are not returned by "mdev -s".
Any scripts launched by mdev can easily figure out whether they've been
called at bootup or in response to a hotplug event, and act accordingly.
A Google search turns up many hits about instructions for automounting
under mdev, and it appears relatively easy.

> What specifically is your objection to udev today?  Is it doing things
> you don't like?  Too big?  Something else?

  Today, it requires an initramfs if /usr is not physically on /.  That
is due in large part to the fact that it has been rolled into the
systemd tarball, and inherited some of systemd's code and limitations,
despite the fact that udev is still a separate binary.

  That's today.  How long before it becomes part of the systemd binary?

-- 
Walter Dnes <waltdnes@waltdnes.org>

Re: [gentoo-dev] Stability of /sys api

[Stelian Ionescu]

[-- Attachment #1: Type: text/plain, Size: 724 bytes --]

On Tue, 2012-05-15 at 18:38 -0400, Walter Dnes wrote:
> On Tue, May 15, 2012 at 11:26:03AM -0700, Greg KH wrote
> > What specifically is your objection to udev today?  Is it doing things
> > you don't like?  Too big?  Something else?
> 
>   Today, it requires an initramfs if /usr is not physically on /.  That
> is due in large part to the fact that it has been rolled into the
> systemd tarball, and inherited some of systemd's code and limitations,
> despite the fact that udev is still a separate binary.

This is absolutely and definitely false. Where did you hear such
nonsense ?


-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Stability of /sys api

[Walter Dnes]

On Wed, May 16, 2012 at 12:44:59AM +0200, Stelian Ionescu wrote
> On Tue, 2012-05-15 at 18:38 -0400, Walter Dnes wrote:
> > On Tue, May 15, 2012 at 11:26:03AM -0700, Greg KH wrote
> > > What specifically is your objection to udev today?  Is it doing things
> > > you don't like?  Too big?  Something else?
> > 
> >   Today, it requires an initramfs if /usr is not physically on /.  That
> > is due in large part to the fact that it has been rolled into the
> > systemd tarball, and inherited some of systemd's code and limitations,
> > despite the fact that udev is still a separate binary.
> 
> This is absolutely and definitely false. Where did you hear such
> nonsense ?

  1) Did you sleep through the /usr and initramfs flamewars?
http://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken

  2) The udev sources have merged into the systemd tarball.  See...
http://article.gmane.org/gmane.linux.hotplug.devel/17392  And note
the date is April 3rd, not April 1st.  If they were really as worried
about compatability as they claim, you wouldn't need to use initramfs
for udev with a separate /usr.

-- 
Walter Dnes <waltdnes@waltdnes.org>

Re: [gentoo-dev] Stability of /sys api

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1440 bytes --]

On Tue, May 15, 2012 at 07:51:03PM -0400, Walter Dnes wrote:
> On Wed, May 16, 2012 at 12:44:59AM +0200, Stelian Ionescu wrote
> > On Tue, 2012-05-15 at 18:38 -0400, Walter Dnes wrote:
> > > On Tue, May 15, 2012 at 11:26:03AM -0700, Greg KH wrote
> > > > What specifically is your objection to udev today?  Is it doing things
> > > > you don't like?  Too big?  Something else?
> > > 
> > >   Today, it requires an initramfs if /usr is not physically on /.  That
> > > is due in large part to the fact that it has been rolled into the
> > > systemd tarball, and inherited some of systemd's code and limitations,
> > > despite the fact that udev is still a separate binary.
> > 
> > This is absolutely and definitely false. Where did you hear such
> > nonsense ?
> 
>   1) Did you sleep through the /usr and initramfs flamewars?
> http://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken
> 
>   2) The udev sources have merged into the systemd tarball.  See...
> http://article.gmane.org/gmane.linux.hotplug.devel/17392  And note
> the date is April 3rd, not April 1st.  If they were really as worried
> about compatability as they claim, you wouldn't need to use initramfs

If you saw my last message on this subject, there is no need to use
initramfs if you don't want to use it.

See the sep-usr use flag on the ~arch version of busybox and the
instructions you get when you turn that on.

William

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] Stability of /sys api

[Arun Raghavan]

On 16 May 2012 05:21, Walter Dnes <waltdnes@waltdnes.org> wrote:
> On Wed, May 16, 2012 at 12:44:59AM +0200, Stelian Ionescu wrote
>> On Tue, 2012-05-15 at 18:38 -0400, Walter Dnes wrote:
>> > On Tue, May 15, 2012 at 11:26:03AM -0700, Greg KH wrote
>> > > What specifically is your objection to udev today?  Is it doing things
>> > > you don't like?  Too big?  Something else?
>> >
>> >   Today, it requires an initramfs if /usr is not physically on /.  That
>> > is due in large part to the fact that it has been rolled into the
>> > systemd tarball, and inherited some of systemd's code and limitations,
>> > despite the fact that udev is still a separate binary.
>>
>> This is absolutely and definitely false. Where did you hear such
>> nonsense ?
>
>  1) Did you sleep through the /usr and initramfs flamewars?
> http://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken

You seem to have missed the bit that this has nothing at all to do with systemd.

-- 
Arun Raghavan
http://arunraghavan.net/
(Ford_Prefect | Gentoo) & (arunsr | GNOME)

Re: [gentoo-dev] Stability of /sys api

[Luca Barbato]

On 15/05/12 21:07, Arun Raghavan wrote:
>>  1) Did you sleep through the /usr and initramfs flamewars?
>> http://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken
> 
> You seem to have missed the bit that this has nothing at all to do with systemd.
> 

I guess the systemd in the url might be have people consider it a sort
of tell-tale. It could had been

http://bluez.org/known_issue/separate-usr-problems

Maybe.

-- 

Luca Barbato
Gentoo/linux
http://dev.gentoo.org/~lu_zero

Re: [gentoo-dev] Stability of /sys api

[Walter Dnes]

  I asked what I thought was a simple developer-type question.  I don't
want this to become a public flamewar.  If anybody wants to discuss the
issue with me further, please email directly to me and not the list.

-- 
Walter Dnes <waltdnes@waltdnes.org>