[gentoo-amd64] Re: Systemd migration: opinion and questions

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Marc Joliet posted on Tue, 24 Feb 2015 21:15:45 +0100 as excerpted:

> Like I mentioned above, my Desktop can *almost* suspend reliably,
> after trying it out once every year or two (it's over 8 years old).
> Mostly it would just not wake back up.  The latest status (before
> systemd) was that the kernel crashed after waking up (but I think
> that was a known bug that was fixed in the meantime).

FWIW... for readers that might be considering multi-device filesystems 
(like btrfs can do if so configured) or mdraid/dmraid/etc type multi-
device backed filesystems...

What I've found with suspend/hibernate over the years and on multiple 
systems, is that while at least one or the other generally works,
that's on the condition of no multi-device mdraid, dmraid, btrfs, etc.
Once you get more than one physical device backing a filesystem, be
that mdraid/ dmraid/etc, or a direct multi-device filesystem such as
btrfs, suspend/ hibernate, or more precisely, the resume, becomes
problematic, because invariably, one device lags the others in
resuming, and the kernel apparently doesn't know how to properly wait
for multiple devices to all resume and stabilize at once.

The result is that the system resumes, but one or more physical devices 
underlying that multi-device filesystem often stabilizes slower than
the others and gets dropped from the raid or whatever.

If it's a raid0, without redundancy, that can mean you just lost it and 
everything on it, period.  With other raid types it's not so bad, but
it does often mean either manual missing-device delete and re-add
(mdraid), or (on btrfs) a quick reboot as btrfs becomes unstable when a
device drops and the system will often freeze or livelock if you don't,
and then a scrub after the reboot brings the device back in, to sync
the updates back to the device that was dropped and brought back in.

So... after finding that about half the time after a suspend or
hibernate and resume cycle you have to either reboot or do manual
system maintenance anyway, pretty soon you learn not to bother with
suspend/ hibernate in the first place, and simply shutdown, and restart
from power- off when you'd otherwise resume.

Back on spinning rust, this used to annoy me greatly, as the boot time 
wasn't bad, but it did mean starting with an empty cache, and losing
that several gigs of cache of much slower spinning rust at the reboot
was / painful/.

While reasonably fast SSDs are still slower than cache, the difference
is 2-3 orders of magnitude smaller, and losing the cache on reboot
isn't the big deal it once was.  Between that and the fact that systemd
bootup is so fast on ssd, full shutdown and restart isn't such a big
deal these days.

But it sure would be nice if the kernel could learn to handle resume
from suspend or hibernate much like it does bootup, using bootwait or
similar kernel commandline option not just at boot, but at resume as
well, so systems that can and do /boot/ multiple devices just fine,
can /resume/ them just fine as well.  Then we'd not have to worry about
such problems.  Oh, well... maybe someday...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman


-- 
Duncan - No HTML messages please; they are filtered as spam.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-amd64] Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 14723 bytes --]

Hi list,

(Normally I ask these types of questions to gentoo-user, but thought I'd try
here for once, especially given the ease with which systemd flamewars erupt on
gentoo-user, compared to the much more civil discussions I've seen here.)

So, at night on Saturday, 14th February, I decided, on a whim, to install
systemd on my Uni laptop.  Everything went well, save for some remaining
questions (see below).  The Gentoo systemd wiki entry was very helpful.

[ It's really strange how that works: I just got the sudden urge to try systemd
  out, for no real reason at all, and couldn't stop myself.  Weird... ]

Heh... I was originally going to send this *before* migrating my desktop, but
it turns out I liked systemd enough to want to switch my desktop to it before I
managed to finish writing this.  And holy crap, Duncan was right: systemd is
*so* *fast* on an SSD, it's just not funny.  It takes *3 seconds* after the
kernel boots for me to get a login screen (for a total of about 5-6 seconds
after the boot loader).  I barely get a chance to see systemd's messages, they
just start zipping by, and then *wham*: login screen.  I am *amazed*!

*ahem*

Anyway, I thought it might be interesting to report other things I did and what
I like about systemd so far.  Due to the length and additional context I'll
split the rest of this email into sections for better readability.  I apologise
for the length; in my defence, this is a fundamental system change, and I
wanted to offer enough context for everything.

== Additional changes on the laptop ==

I replaced laptop-mode-tools with tlp from the overlay of the same name (not
directly available via layman, but there's a bug on b.g.o that links to the
GitHub repo, which has installation instructions).  Since I controlled the
backlight with lmt, and tlp doesn't support that, I had to find a replacement
for that feature.  I ultimately decided on power-backlight, which I found via
the Arch Linux wiki.  I wrote an ebuild and put it in my personal overlay
(http://sourceforge.net/p/mjolietoverlay/code/ci/master/tree/app-laptop/power-backlight/),
and so far everything is working fine.

This might seem like a pointless change, but Arch switched to it, too, and
doesn't even package lmt anymore (albeit only since late 2014), so it seems
like the way forward.

== Additional changes on both systems ==

I uninstalled acpid, since the Gentoo Wiki page on systemd mentions that it
might not be necessary (see the question below).  Being able to do this was
another reason I switched to tlp, since it doesn't depend on acpid.  So far I
have yet to notice anything amiss.

I also uninstalled tmpwatch, since systemd has a built-in service for doing the
same thing (systemd-tmpfiles).

== Things I have *not* gotten rid of (yet) ==

Fcron is still around, mainly because packages might rely on it being there
(e.g., man-db and mlocate install files there), but also because I haven't
researched systemd timers yet.

I plan on uninstalling syslog-ng, but haven't done so yet.  I simply feel
better waiting a bit, even though I don't run it anymore.  Man, I feel silly
after typing that...

== Network migration on my desktop ==

My desktop has a somewhat more complicated network setup than the laptop (which
uses NetworkManager).  The wan0 network device is no problem, but I also have a
bridge with one physical device connected to it, and one dummy device.

Thus, I had to migrate my netifrc based configuration.  Due to word of mouth, I
decided on netctl.  I originally ignored systemd-networkd because I keep
hearing that it's only for simple networks, but after looking at the man page
it appears that it can do (almost) everything that I needed it to, although I'm
not sure about dummy device support.  I need that for MATLAB, which stupidly
requires the presence of an interface named eth0, whose MAC address it verifies
during its licence check.  This became a problem after renaming my network
devices following the news entry 2013-03-29-udev-upgrade ("Upgrading udev to
version >=200").

What surprised me was that netifrc doesn't seem to integrate dummy devices
properly, i.e., it doesn't seem to be possible to rename them in
/etc/conf.d/net directly.  I implemented that via an appropriate call to ip in
an /etc/local.d file, and had "net.eth0" depend on "local".  So /etc/conf.d/net
looked like this:

    config_eth0="172.16.1.1/24"
    mac_eth0="00:18:f3:97:17:72"
    rc_net_eth0_provide="!net"
    rc_net_eth0_need="local"

And the local.d script executed

    ip link set dev dummy0 name eth0

In comparison, the netctl configuration has everything in one place:

    # cat /etc/netctl/dummy
    Description='A dummy interface (for MATLAB)'
    Interface=eth0
    Connection=dummy
    IP=static
    Address=172.16.1.1
    IPCustom=( 'link set dev eth0 address 00:18:f3:97:17:72' )

It also takes care of loading the dummy kernel module, so I don't don't need an
/etc/modules-load.d/ entry, whereas with netifrc/OpenRC I needed to load it
manually via /etc/conf./modules (though maybe that has changed?).

== Stuff I liked ==

If you want to skip this, the questions are in the next section.

=== Pleasant surprises ===

Some surprising things worked out of the box, for example all /etc/local.d/
files are dynamically converted to units at run-time and executed.  I hardly
had to migrate anything.  The one exception was chrony, where I had to override
the service file to add extra command line flags that don't have corresponding
configuration file entries.

On my desktop I had to add network dependencies to some socket units so that
they wouldn't start before the network they're supposed to listen on is fully
configured (see the question below).  Otherwise, it just had a few more things
that needed doing, but that is the nature of running services on it (dovecot,
postfix, and samba).

=== The Journal ===

I like the journal.  No, sorry, I *really* *really* like the journal. I like
the filters you can apply (e.g., limiting the output by unit via the -u flag),
the priority system, the automatic use of ACLs so that you can see your own
user's logs. I also think that it's a nice detail that "systemctl status -u"
will show the last lines of log output. This actually helped me at work :-) .

=== Socket activation ===

I especially learned to like it after reading the early "systemd for admins"
articles, which made the implications of the design more clear.  I definitely
like the ability to have a service start on-demand instead of spending 99.99%
of its time idle.  Finally, I also like the simplicity of the C API from
systemd/sd-daemon.h.

=== Miscellanea ===

Some things work better, e.g., closing the laptop's lid suspends the laptop
automatically without me having to configure anything.  My desktop is *almost*
there, see the questions below.

Systemd can restart itself, e.g., for upgrades.  Nice!

Systemd-analyze is really nice.

Systemd exposes various system features that require extra work to set up
otherwise.  The example that made me think of this is "systemctl kexec", but
from reading Leannart Poettering's blog series, this is one of the goals of
systemd.  I can definitely sympathise with that.

== Questions ===

=== ACPID: needed or not? ===

Does acpid provide anything that systemd does not, and if so, what kind of
"conflicts" might I see?  The Gentoo Wiki page says that acpid is likely not
needed.

FWIW, I already unmerged it and have not noticed any missing functionality,
even after over a week of regular usage.  So I'm tending towards "no, not
needed".

=== Timers ===

Can a systemd timer depend on a mount point such that it waits until the mount
point exists before running?  Or will it fail after a timeout?  I want to
research this myself, but haven't gotten around to it yet.

The problem I have is that my external HDD does not come up properly on cold
boot, so that I have to unplug it and plug it back in in order for the kernel
to fully initialise it and for it to mount, which is problematic for my backup
fcron jobs, since they have &bootrun set. This means that the backup script
will fail, unless I'm a) fast enough at re-plugging the HDD, and b) fast enough
at logging in (so that my automounter mounts the HDD.  I will then have to
manually re-run it (i.e., fcrondyn -x "run <ID>") or wait for the next time
it's supposed to run.

Naturally, I would like a more robust system than that, and hope that systemd
timers can make my life easier here.

=== User units ===

I would like to convert some programs I start in .xprofile to units that are
started by my users's systemd instance.  I started off with mpd, but it doesn't
start automatically ("systemctl --user start mpd" works fine, though), even
though it's enabled:

    % systemctl --user status mpd
    ● mpd.service - Music Player Daemon
       Loaded: loaded (/usr/lib64/systemd/system/mpd.service; enabled)
       Active: active (running) since Di 2015-02-24 19:39:46 CET; 1h 6min ago
     Main PID: 1091 (mpd)
       CGroup: /user.slice/user-1000.slice/user@1000.service/mpd.service
               └─1091 /usr/bin/mpd --no-daemon

    Feb 24 19:39:46 marcec systemd[384]: Started Music Player Daemon.
    [...]

Also:

    % tree .config/systemd/
    .config/systemd/
    └── user
        ├── mpd.service -> /usr/lib64/systemd/system/mpd.service
        └── multi-user.target.wants
            └── mpd.service -> /home/marcec/.config/systemd/user/mpd.service

Is the symlink the problem?  Do I have to create an actual file?  Is the
target.wants wrong?

=== Suspend on the desktop ===

Like I mentioned above, my Desktop can *almost* suspend reliably, after trying
it out once every year or two (it's over 8 years old).  Mostly it would just
not wake back up.  The latest status (before systemd) was that the kernel
crashed after waking up (but I think that was a known bug that was fixed in the
meantime).

Now with systemd, it wakes back up properly, but one of the soundcard drivers
(ice1724) is apparently unreliable, so that sound stops working after wakeup.
Plus, access to the soundcard was blocked by rtkit.  I believe what one would
normally do is unload the module before suspending, but the only way I could
find to tell systemd to do that requires creating a file in /usr (as done here,
for example: http://forums.fedoraforum.org/showthread.php?t=294065).  Is there
a better way?

=== Dovecot and socket activation ===

Dovecot gave me problems with socket activation, I kept getting errors like the
following:

    Feb 20 22:58:19 marcec dovecot[6500]: master: Panic: io_add(0x1) called
    twice fd=4, callback=0x40a970 -> 0x40a970 Feb 20 22:58:19 marcec
    dovecot[6500]: master: Error: Raw
    backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x6accf) [0x7f42b9300ccf]
    -> /usr/lib64/dovecot/libdovecot.so.0(i_syslog_fatal_handler

I don't know who's at fault here, systemd or dovecot, but I had to simplify the
configuration a bit:

    diff --git a/dovecot/conf.d/10-master.conf b/dovecot/conf.d/10-master.conf
    index ddbde28..c0ca867 100644
    --- a/dovecot/conf.d/10-master.conf
    +++ b/dovecot/conf.d/10-master.conf
    @@ -16,11 +16,11 @@
     
     service imap-login {
       inet_listener imap {
    -    address = ::1, 127.0.0.1, 172.16.0.1
    +    address = ::1, 172.16.0.1
         port = 10087
       }
       inet_listener imaps {
    -    address = ::1, 127.0.0.1, 172.16.0.1
    +    address = ::1, 172.16.0.1
         port = 10887
         #ssl = yes
       }
    diff --git a/systemd/system/dovecot.socket.d/sockets.conf
    b/systemd/system/dovecot.socket.d/sockets.conf index bcc29c6..368d461 100644
    --- a/systemd/system/dovecot.socket.d/sockets.conf
    +++ b/systemd/system/dovecot.socket.d/sockets.conf
    @@ -1,8 +1,6 @@
     [Socket]
     ListenStream=
    -ListenStream=127.0.0.1:10087
     ListenStream=172.16.0.1:10087
     ListenStream=[::1]:10087
    -ListenStream=127.0.0.1:10887
     ListenStream=172.16.0.1:10887
     ListenStream=[::1]:10887

Note that it also worked if I removed 127.0.0.1 from imap, and 172.16.0.1 from
imaps (or, IIRC, vice versa).  I'm fine with the configuration being like this,
but I'm nonetheless baffled by the necessity.  I couldn't even find a bug
report or ML post.

=== Depending on a specific network interface ===

Some socket units failed to start at first, due to "resource" errors.  So I
made them depend on netctl@bridge via *.d/requires.conf files like so:

    [Unit]
    Requires=netctl@bridge.service
    After=netctl@bridge.service

That fixed the errors, but is it the correct way to depend on that interface
(ignoring the fact that I could have put symlinks at the right place instead)?

=== dmesg and syslog ===

On the laptop I didn't immediately stop running syslog-ng.  Following a thread
on gentoo-user about the journal complaining about messages not being forwarded
to syslog-ng, I got curious and checked the laptop today.

What I did was run

    diff -U8 <(zcat /var/log/messages-*) <(journalctl)

and look for differences in the time frame where both were running (note
that /var/log/messages is empty, so I only checked the rotated logs).

First of all, while I clearly remember seeing similar messages about "missing
messages", I couldn't find any in either the journal or syslogs output.  Maybe
I mixed them up with "Forward time jump detected" messages from chrony?  But
those are too rare, unless I noticed one right after resuming from suspend and
got mixed up then?  I think that I would like confirmation that those messages
are actually logged properly, and where.

However, I did find something unexpected.  I haven't looked deeply yet, but I
noticed some dmesg entries not ending up in the journal (i.e., a bunch of "-"
lines in-between the "+" lines in the diff output). Did I do something wrong, or
is this known behaviour when running the journal and syslog-ng simultaneously?
I'll look more closely tomorrow, but wondered if anybody here noticed anything
similar.

== The End ==

*phew*

Of course, some of the complications will go away once I've gotten around to
turning my desktop back into a pure desktop and not the desktop/server/router
hybrid it is now, but that will have to wait.

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Systemd migration: opinion and questions

[Rich Freeman]

Lots of stuff here.  I'm only replying where I think I can add value.

On Tue, Feb 24, 2015 at 3:15 PM, Marc Joliet <marcec@gmx.de> wrote:
>
> == Things I have *not* gotten rid of (yet) ==
>
> Fcron is still around, mainly because packages might rely on it being there
> (e.g., man-db and mlocate install files there), but also because I haven't
> researched systemd timers yet.

My fault.  :)  I have sys-process/systemd-cron in my rich0 overlay but
I've been too lazy to get it into the main tree (I need to check to
see what units if any I had to tweak from what gets installed and
merge those back into the package).  This is a set of timer/service
units that will run all the /etc/cron.*/ jobs.  They'll run in series
and not in parallel as they would if you turned them into individual
timer scripts, but it is a great way to just integrate the scripts
packages already are providing.  It also has a crontab generator, but
I don't think it works with fcrontab so I have not tested this.
Generators are just one of those little things I love about systemd.

>
> I plan on uninstalling syslog-ng, but haven't done so yet.  I simply feel
> better waiting a bit, even though I don't run it anymore.  Man, I feel silly
> after typing that...

Yeah, took me a while to uninstall it, but I don't find much value in
running both.

>
> == Network migration on my desktop ==
>
> My desktop has a somewhat more complicated network setup than the laptop (which
> uses NetworkManager).  The wan0 network device is no problem, but I also have a
> bridge with one physical device connected to it, and one dummy device.  ...
> I originally ignored systemd-networkd because I keep
> hearing that it's only for simple networks, but after looking at the man page
> it appears that it can do (almost) everything that I needed it to, although I'm
> not sure about dummy device support.

Here is what I'm doing for a bridge - I can't vouch for dummy devices
but I wouldn't be surprised if it works (remember, these are the guys
doing udev too):
cd /etc/systemd/network/

cat brkvm.netdev
[NetDev]
Name=brkvm
Kind=bridge


cat brkvm.network
[Match]
Name=brkvm

[Network]
DNS=192.168.0.5
Address=192.168.0.5/24
Gateway=192.168.0.21


cat eth-bridge.network
[Match]
Name=e*

[Network]
Bridge=brkvm


If you're using persistent names you might need to tweak the Name in
the last one slightly so that it matches whatever you're using.

>
> === Timers ===
>
> Can a systemd timer depend on a mount point such that it waits until the mount
> point exists before running?  Or will it fail after a timeout?  I want to
> research this myself, but haven't gotten around to it yet.

So, timer units are units, and units can have dependencies, and mounts
can be dependencies since mounts are units.  However, if you set the
dependency on the timer itself, then the timer won't start running
until the mount exists.  You probably want the depencency to be on the
service started by the timer (so the timer is watching the clock, but
the service won't start without the mount).  If you set a
Requires=foo.mount and After=foo.mount, then the service shouldn't run
unless foo.mount is available.  I suspect systemd will attempt to
mount the filesystem when it runs the service, and you'll get units in
the failed state if that doesn't work.

However, I haven't tested any of this.  I suspect it wouldn't take
much to work this out.  I have a mount dependency in one of my
services.  Just look at the mount units in /run/systemd/generator for
the name of the mount unit systemd is creating from fstab.

> === Depending on a specific network interface ===
>
> Some socket units failed to start at first, due to "resource" errors.  So I
> made them depend on netctl@bridge via *.d/requires.conf files like so:
>
>     [Unit]
>     Requires=netctl@bridge.service
>     After=netctl@bridge.service
>
> That fixed the errors, but is it the correct way to depend on that interface
> (ignoring the fact that I could have put symlinks at the right place instead)?

Following my networkd scripts above, I suspect that something like
this /might/ work but I haven't tested it:
Requires=sys-subsystem-net-devices-brkvm.device
After=sys-subsystem-net-devices-brkvm.device

Again, another example of the udev integration - you get device units
for devices.  That also lets you do things like start a service when a
device is activated, have dependencies, etc.  You should look at the
list of units sometime - you can also do things like run a service
when a file is modified (no need to have a helper program watching it
with fnotify, etc).

Also, I think Requires basically implies Wants as well, so if that
network device doesn't exist and the service tries to start, systemd
might try to create that device, if possible.

Now, the device existing does not necessarily mean that it has an IP,
working DNS, etc.  I don't know if there is any way to have a
dependency on a specific configured network.  That might be a great
question to ask the systemd devs.

-- 
Rich

Re: [gentoo-amd64] Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 8184 bytes --]

Am Tue, 24 Feb 2015 16:44:59 -0500
schrieb Rich Freeman <rich0@gentoo.org>:

> Lots of stuff here.  I'm only replying where I think I can add value.
> 
> On Tue, Feb 24, 2015 at 3:15 PM, Marc Joliet <marcec@gmx.de> wrote:
> >
> > == Things I have *not* gotten rid of (yet) ==
> >
> > Fcron is still around, mainly because packages might rely on it being there
> > (e.g., man-db and mlocate install files there), but also because I haven't
> > researched systemd timers yet.
> 
> My fault.  :)  I have sys-process/systemd-cron in my rich0 overlay but
> I've been too lazy to get it into the main tree (I need to check to
> see what units if any I had to tweak from what gets installed and
> merge those back into the package).  This is a set of timer/service
> units that will run all the /etc/cron.*/ jobs.  They'll run in series
> and not in parallel as they would if you turned them into individual
> timer scripts, but it is a great way to just integrate the scripts
> packages already are providing.  It also has a crontab generator, but
> I don't think it works with fcrontab so I have not tested this.
> Generators are just one of those little things I love about systemd.

That sounds great!  I'll be looking forward to this :) .

(Although, sadly, it does appear as if ebuilds getting stuck in overlays happens
much too often these days.)

> >
> > I plan on uninstalling syslog-ng, but haven't done so yet.  I simply feel
> > better waiting a bit, even though I don't run it anymore.  Man, I feel silly
> > after typing that...
> 
> Yeah, took me a while to uninstall it, but I don't find much value in
> running both.

Precisely.  It's more a "but what if...!" kind of worry.

> >
> > == Network migration on my desktop ==
> >
> > My desktop has a somewhat more complicated network setup than the laptop (which
> > uses NetworkManager).  The wan0 network device is no problem, but I also have a
> > bridge with one physical device connected to it, and one dummy device.  ...
> > I originally ignored systemd-networkd because I keep
> > hearing that it's only for simple networks, but after looking at the man page
> > it appears that it can do (almost) everything that I needed it to, although I'm
> > not sure about dummy device support.
> 
> Here is what I'm doing for a bridge - I can't vouch for dummy devices
> but I wouldn't be surprised if it works (remember, these are the guys
> doing udev too):
> cd /etc/systemd/network/
> 
> cat brkvm.netdev
> [NetDev]
> Name=brkvm
> Kind=bridge
> 
> 
> cat brkvm.network
> [Match]
> Name=brkvm
> 
> [Network]
> DNS=192.168.0.5
> Address=192.168.0.5/24
> Gateway=192.168.0.21
> 
> 
> cat eth-bridge.network
> [Match]
> Name=e*
> 
> [Network]
> Bridge=brkvm
> 
> 
> If you're using persistent names you might need to tweak the Name in
> the last one slightly so that it matches whatever you're using.

That looks nice, I like that the bridge device is distinct from the bridge
network, is distinct from other devices attached to it.  This might help in
getting unit dependencies right. I'll try it out this weekend.

My primary concern with networkd would be that it is very new, and thus not as
well tested as other solutions.  While netctl is also fairly new, and
I'm using that with no problems, it uses well-known DHCP clients, while it
appears that networkd implements its own.

But I'll give it a whirl anyway ;) .

> > === Timers ===
> >
> > Can a systemd timer depend on a mount point such that it waits until the mount
> > point exists before running?  Or will it fail after a timeout?  I want to
> > research this myself, but haven't gotten around to it yet.
> 
> So, timer units are units, and units can have dependencies, and mounts
> can be dependencies since mounts are units.  However, if you set the
> dependency on the timer itself, then the timer won't start running
> until the mount exists.  You probably want the depencency to be on the
> service started by the timer (so the timer is watching the clock, but
> the service won't start without the mount).

Wait, so the timer won't start watching the clock until its dependencies are
met (i.e, the mount point appears)?  Is that what you mean?  Because that might
be more in line with what I want (though I'm not sure yet).

> If you set a
> Requires=foo.mount and After=foo.mount, then the service shouldn't run
> unless foo.mount is available.  I suspect systemd will attempt to
> mount the filesystem when it runs the service, and you'll get units in
> the failed state if that doesn't work.
>
> However, I haven't tested any of this.  I suspect it wouldn't take
> much to work this out.  I have a mount dependency in one of my
> services.  Just look at the mount units in /run/systemd/generator for
> the name of the mount unit systemd is creating from fstab.

Right, so IIUC, I would have a oneshot service that does the backup, and the timer
runs that, and of course the timer can depend on the mount point.  And if the
mount point doesn't exist, then the service started by the timer will fail.

What I would prefer to have is a timer that only runs if *both* the time *and*
mount conditions are met.  Skimming the man page, this does not seem possible.
I suppose it would be nice if timers learned "conditions" on which they should
wait in addition to the time condition, but maybe that's outside the scope of
systemd?

But my confidence that at the very least something better than what I have now
will be possible, I'll just have to take the time and read the man pages
properly.

> > === Depending on a specific network interface ===
> >
> > Some socket units failed to start at first, due to "resource" errors.  So I
> > made them depend on netctl@bridge via *.d/requires.conf files like so:
> >
> >     [Unit]
> >     Requires=netctl@bridge.service
> >     After=netctl@bridge.service
> >
> > That fixed the errors, but is it the correct way to depend on that interface
> > (ignoring the fact that I could have put symlinks at the right place instead)?
> 
> Following my networkd scripts above, I suspect that something like
> this /might/ work but I haven't tested it:
> Requires=sys-subsystem-net-devices-brkvm.device
> After=sys-subsystem-net-devices-brkvm.device
> 
> Again, another example of the udev integration - you get device units
> for devices.  That also lets you do things like start a service when a
> device is activated, have dependencies, etc.  You should look at the
> list of units sometime - you can also do things like run a service
> when a file is modified (no need to have a helper program watching it
> with fnotify, etc).

Right, I really do like that about systemd, too.  It's much more thorough than
other systems (in several ways).  The mount point units generated from fstab
are one example, but I also noticed that each script in /etc/local.d/ gets the
same treatment and is exposed as its own unit.

The more I use systemd, the more I realise why it made sense to merge the udev
project into systemd: they both play off each others capabilities quite
nicely (udev giving systemd extra information and events to act upon, and
systemd reacting to udev's events and doing the appropriate thing).

Diving into systemd on a whim really was the best way to see what all the fuss
was about, and so far it was worth it.

> Also, I think Requires basically implies Wants as well, so if that
> network device doesn't exist and the service tries to start, systemd
> might try to create that device, if possible.
> 
> Now, the device existing does not necessarily mean that it has an IP,
> working DNS, etc.  I don't know if there is any way to have a
> dependency on a specific configured network.  That might be a great
> question to ask the systemd devs.

Sure, if nobody here gives an authoritative answer, I'll ask there.  People:
you have until the weekend ;) .

Thanks to everyone for the responses so far!
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Systemd migration: opinion and questions

[Rich Freeman]

On Wed, Feb 25, 2015 at 2:50 AM, Marc Joliet <marcec@gmx.de> wrote:
> Am Tue, 24 Feb 2015 16:44:59 -0500
> schrieb Rich Freeman <rich0@gentoo.org>:
>
>> > === Timers ===
>> >
>> > Can a systemd timer depend on a mount point such that it waits until the mount
>> > point exists before running?  Or will it fail after a timeout?  I want to
>> > research this myself, but haven't gotten around to it yet.
>>
>> So, timer units are units, and units can have dependencies, and mounts
>> can be dependencies since mounts are units.  However, if you set the
>> dependency on the timer itself, then the timer won't start running
>> until the mount exists.  You probably want the depencency to be on the
>> service started by the timer (so the timer is watching the clock, but
>> the service won't start without the mount).
>
> Wait, so the timer won't start watching the clock until its dependencies are
> met (i.e, the mount point appears)?  Is that what you mean?  Because that might
> be more in line with what I want (though I'm not sure yet).

If you set the dependency on the timer, then the timer doesn't start
watching the clock until they're met.  If you set the dependency on
the service started by the timer then it will watch the clock but not
launch the service if the dependency isn't met.  You can set the
dependency in either or both places.  The timer and the service are
both units.

>
>> If you set a
>> Requires=foo.mount and After=foo.mount, then the service shouldn't run
>> unless foo.mount is available.  I suspect systemd will attempt to
>> mount the filesystem when it runs the service, and you'll get units in
>> the failed state if that doesn't work.
>>
>> However, I haven't tested any of this.  I suspect it wouldn't take
>> much to work this out.  I have a mount dependency in one of my
>> services.  Just look at the mount units in /run/systemd/generator for
>> the name of the mount unit systemd is creating from fstab.
>
> Right, so IIUC, I would have a oneshot service that does the backup, and the timer
> runs that, and of course the timer can depend on the mount point.  And if the
> mount point doesn't exist, then the service started by the timer will fail.
>
> What I would prefer to have is a timer that only runs if *both* the time *and*
> mount conditions are met.  Skimming the man page, this does not seem possible.
> I suppose it would be nice if timers learned "conditions" on which they should
> wait in addition to the time condition, but maybe that's outside the scope of
> systemd?

I think if you just set the dependency on the service you'll get the
behavior you desire.  Systemd will try to mount the backup filesystem,
and if that fails it won't run the backup.

You can set conditions on units as well, like only running if they're
on AC power or on amd64 or to run one unit the first time you start a
service and a different unit every other time.  Some of that was
designed to implement some of the stateless system features they're
adding to systemd.

-- 
Rich

Re: [gentoo-amd64] Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 3456 bytes --]

Am Wed, 25 Feb 2015 07:01:59 -0500
schrieb Rich Freeman <rich0@gentoo.org>:

> On Wed, Feb 25, 2015 at 2:50 AM, Marc Joliet <marcec@gmx.de> wrote:
> > Am Tue, 24 Feb 2015 16:44:59 -0500
> > schrieb Rich Freeman <rich0@gentoo.org>:
> >
> >> > === Timers ===
> >> >
> >> > Can a systemd timer depend on a mount point such that it waits until the mount
> >> > point exists before running?  Or will it fail after a timeout?  I want to
> >> > research this myself, but haven't gotten around to it yet.
> >>
> >> So, timer units are units, and units can have dependencies, and mounts
> >> can be dependencies since mounts are units.  However, if you set the
> >> dependency on the timer itself, then the timer won't start running
> >> until the mount exists.  You probably want the depencency to be on the
> >> service started by the timer (so the timer is watching the clock, but
> >> the service won't start without the mount).
> >
> > Wait, so the timer won't start watching the clock until its dependencies are
> > met (i.e, the mount point appears)?  Is that what you mean?  Because that might
> > be more in line with what I want (though I'm not sure yet).
> 
> If you set the dependency on the timer, then the timer doesn't start
> watching the clock until they're met.  If you set the dependency on
> the service started by the timer then it will watch the clock but not
> launch the service if the dependency isn't met.  You can set the
> dependency in either or both places.  The timer and the service are
> both units.

OK, I think I got it.

> >
> >> If you set a
> >> Requires=foo.mount and After=foo.mount, then the service shouldn't run
> >> unless foo.mount is available.  I suspect systemd will attempt to
> >> mount the filesystem when it runs the service, and you'll get units in
> >> the failed state if that doesn't work.
> >>
> >> However, I haven't tested any of this.  I suspect it wouldn't take
> >> much to work this out.  I have a mount dependency in one of my
> >> services.  Just look at the mount units in /run/systemd/generator for
> >> the name of the mount unit systemd is creating from fstab.
> >
> > Right, so IIUC, I would have a oneshot service that does the backup, and the timer
> > runs that, and of course the timer can depend on the mount point.  And if the
> > mount point doesn't exist, then the service started by the timer will fail.
> >
> > What I would prefer to have is a timer that only runs if *both* the time *and*
> > mount conditions are met.  Skimming the man page, this does not seem possible.
> > I suppose it would be nice if timers learned "conditions" on which they should
> > wait in addition to the time condition, but maybe that's outside the scope of
> > systemd?
> 
> I think if you just set the dependency on the service you'll get the
> behavior you desire.  Systemd will try to mount the backup filesystem,
> and if that fails it won't run the backup.
> 
> You can set conditions on units as well, like only running if they're
> on AC power or on amd64 or to run one unit the first time you start a
> service and a different unit every other time.  Some of that was
> designed to implement some of the stateless system features they're
> adding to systemd.

Right, I'll have a look at them.

Thanks
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 5920 bytes --]

Am Wed, 25 Feb 2015 19:25:08 +0100
schrieb Marc Joliet <marcec@gmx.de>:

> Am Wed, 25 Feb 2015 07:01:59 -0500
> schrieb Rich Freeman <rich0@gentoo.org>:
> 
> > On Wed, Feb 25, 2015 at 2:50 AM, Marc Joliet <marcec@gmx.de> wrote:
> > > Am Tue, 24 Feb 2015 16:44:59 -0500
> > > schrieb Rich Freeman <rich0@gentoo.org>:
> > >
> > >> > === Timers ===
> > >> >
> > >> > Can a systemd timer depend on a mount point such that it waits until the mount
> > >> > point exists before running?  Or will it fail after a timeout?  I want to
> > >> > research this myself, but haven't gotten around to it yet.
> > >>
> > >> So, timer units are units, and units can have dependencies, and mounts
> > >> can be dependencies since mounts are units.  However, if you set the
> > >> dependency on the timer itself, then the timer won't start running
> > >> until the mount exists.  You probably want the depencency to be on the
> > >> service started by the timer (so the timer is watching the clock, but
> > >> the service won't start without the mount).
> > >
> > > Wait, so the timer won't start watching the clock until its dependencies are
> > > met (i.e, the mount point appears)?  Is that what you mean?  Because that might
> > > be more in line with what I want (though I'm not sure yet).
> > 
> > If you set the dependency on the timer, then the timer doesn't start
> > watching the clock until they're met.  If you set the dependency on
> > the service started by the timer then it will watch the clock but not
> > launch the service if the dependency isn't met.  You can set the
> > dependency in either or both places.  The timer and the service are
> > both units.
> 
> OK, I think I got it.

I finally looked at this more closely yesterday.  No, dependencies don't do
what I want.  If a dependency is not met, a unit goes into a failed state.
Since the problem is that my external drive doesn't show up properly until I
unplug it and plug it back in (well, strictly speaking, its device name shows
up, but not its partitions), that won't work because that would require manual
intervention, which I would very much like to avoid.

> > >> If you set a
> > >> Requires=foo.mount and After=foo.mount, then the service shouldn't run
> > >> unless foo.mount is available.  I suspect systemd will attempt to
> > >> mount the filesystem when it runs the service, and you'll get units in
> > >> the failed state if that doesn't work.

Exactly, setting a dependency on a mount point will make systemd attempt to
mount the file system before starting the unit.  Its the fact that it goes into
a failed state if that attempt fails that's the problem.  Again: manual
intervention.

> > >> However, I haven't tested any of this.  I suspect it wouldn't take
> > >> much to work this out.  I have a mount dependency in one of my
> > >> services.  Just look at the mount units in /run/systemd/generator for
> > >> the name of the mount unit systemd is creating from fstab.
> > >
> > > Right, so IIUC, I would have a oneshot service that does the backup, and the timer
> > > runs that, and of course the timer can depend on the mount point.  And if the
> > > mount point doesn't exist, then the service started by the timer will fail.
> > >
> > > What I would prefer to have is a timer that only runs if *both* the time *and*
> > > mount conditions are met.  Skimming the man page, this does not seem possible.
> > > I suppose it would be nice if timers learned "conditions" on which they should
> > > wait in addition to the time condition, but maybe that's outside the scope of
> > > systemd?
> > 
> > I think if you just set the dependency on the service you'll get the
> > behavior you desire.  Systemd will try to mount the backup filesystem,
> > and if that fails it won't run the backup.
> > 
> > You can set conditions on units as well, like only running if they're
> > on AC power or on amd64 or to run one unit the first time you start a
> > service and a different unit every other time.  Some of that was
> > designed to implement some of the stateless system features they're
> > adding to systemd.
> 
> Right, I'll have a look at them.

The problem with conditions (as they exist in systemd currently) is the same as
with dependencies: the unit does not wait until the condition is met, but
immediately stops (only that it doesn't enter a failed state).  I mean, this is
what conditions in systemd are *supposed* to do, and they do they're
designated job, but I would like the timer to wait until the condition is met
and *then* run the job. I.e., I want a *delay*.

An example of where something like this exists is fcron's lavg* options, where
the jobs is delayed until the specified load average is reached.  I want this,
only for mount points.

Another possibility would be something akin to PartOf that would additionally
link two units at *startup*, i.e., the depending unit starts when its
dependency appears.  Then the timers would come and go along with the mount
point, and at bootup, when the drive doesn't show up properly, once I get it
going the timers show up and elapse appropriately.  However, I'm not sure
whether that would be a good/robust system design (e.g., would that mask error
modes I care about?).

*sigh*

Maybe I'm over-thinking this.

Anyway, what I ended up doing is setting Restart=on-failure with appropriate
intervals so that I get a five minute window to unplug the drive and plug it
back in.  My backup script already returns appropriate error codes, so this
just worked.

I'm not entirely happy, but it's still better (in various ways) than what I had
with fcron, despite the greater verbosity of systemd timers compared to crontab
entries.

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Systemd migration: opinion and questions

[Rich Freeman]

On Sun, Mar 1, 2015 at 7:48 AM, Marc Joliet <marcec@gmx.de> wrote:
>
> The problem with conditions (as they exist in systemd currently) is the same as
> with dependencies: the unit does not wait until the condition is met, but
> immediately stops (only that it doesn't enter a failed state).  I mean, this is
> what conditions in systemd are *supposed* to do, and they do they're
> designated job, but I would like the timer to wait until the condition is met
> and *then* run the job. I.e., I want a *delay*.

The timer keeps running if you set the dependency on the service.  So,
next time the timer runs, it will try again.  You might want to just
set an hourly job and have it check for a successful run in the last
day or whatever.

I don't know if restart=on-failure or on-abnormal applies if the
service fails to run in the first place.  You might try setting that
with a suitable restartsec=### setting.  That might be a better way to
get the behavior you want if it works.

-- 
Rich

Re: [gentoo-amd64] Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 3448 bytes --]

Am Sun, 1 Mar 2015 08:34:19 -0500
schrieb Rich Freeman <rich0@gentoo.org>:

> On Sun, Mar 1, 2015 at 7:48 AM, Marc Joliet <marcec@gmx.de> wrote:
> >
> > The problem with conditions (as they exist in systemd currently) is the same as
> > with dependencies: the unit does not wait until the condition is met, but
> > immediately stops (only that it doesn't enter a failed state).  I mean, this is
> > what conditions in systemd are *supposed* to do, and they do they're
> > designated job, but I would like the timer to wait until the condition is met
> > and *then* run the job. I.e., I want a *delay*.
> 
> The timer keeps running if you set the dependency on the service.  So,
> next time the timer runs, it will try again.  You might want to just
> set an hourly job and have it check for a successful run in the last
> day or whatever.
> 
> I don't know if restart=on-failure or on-abnormal applies if the
> service fails to run in the first place.  You might try setting that
> with a suitable restartsec=### setting.  That might be a better way to
> get the behavior you want if it works.

I think I tried that (with a Requisite dependency, because I didn't want the
drive to be mounted automatically by the service).  I'm certain that it did not
work, and that it could not.  Specifically, what I remember seeing is the unit
trying to start and immediately failing due to the missing dependency.

Which makes sense, because Restart= applies to the program being executed
(i.e., processes started by Exec*), not the service unit itself. If the failure
happens before that (e.g., a required dependency is not met) then it doesn't
even get to the point where it can check a return code, so it *cannot* apply.

So the way I'm using Restart= is about as good as it gets.

But actually, while putting off finishing this email, I came to the realisation
that I could of course use WantedBy= in [Install], and following that idea I
ended up with the following [Unit] and [Install] sections:

    [Unit]
    Description=Run hourly backups (timer)
    Requisite=media-MARCEC_BACKUP.mount
    BindsTo=media-MARCEC_BACKUP.mount
    After=media-MARCEC_BACKUP.mount

    [Install]
    WantedBy=timers.target
    WantedBy=media-MARCEC_BACKUP.mount

So the mount point and the timer both depend on each other (with the mount unit
starting first), so the mount tries to start the timer and vice versa, but only
the timer fails if the mount point doesn't exist.  And the BindsTo sees to it
that the timer disappears if the mount does.  Preliminary tests show that it
works: unmounting with udiskie shows that the backup timer disappears, and it
re-appears after mounting the file system again.

I'll have to see what the behaviour is during a cold boot.  The Requisite
dependency on the mount point *should* make the Restart line in the
corresponding service obsolete, since the WantedBy should make the timer start
once the mount point shows up.  I could probably alternatively use
ConditionPathIsMountPoint instead, but I think I prefer the timer failing.

I also have to admit here that I am still not sure *what* I want (sorry), so I
think I'll just have stop fiddling with this for now and see how this works out
in practice.

Regardless: thoughts?

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Systemd migration: opinion and questions

[Rich Freeman]

On Sun, Mar 1, 2015 at 1:20 PM, Marc Joliet <marcec@gmx.de> wrote:
>
> Regardless: thoughts?

I'd probably just do this:
> Am Sun, 1 Mar 2015 08:34:19 -0500
> schrieb Rich Freeman <rich0@gentoo.org>:
>>
>> The timer keeps running if you set the dependency on the service.  So,
>> next time the timer runs, it will try again.  You might want to just
>> set an hourly job and have it check for a successful run in the last
>> day or whatever.
>>

You could of course trigger this from either the mount or hourly.
Anytime you mount the drive or every hour systemd will run the
service, and the service will see if it managed to do a backup/etc in
the last day/week/whatever, and then run if appropriate.

-- 
Rich

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Rich Freeman posted on Sun, 01 Mar 2015 14:13:53 -0500 as excerpted:

> On Sun, Mar 1, 2015 at 1:20 PM, Marc Joliet <marcec@gmx.de> wrote:
>>
>> Regardless: thoughts?
> 
> I'd probably just do this:
>> Am Sun, 1 Mar 2015 08:34:19 -0500 schrieb Rich Freeman
>> <rich0@gentoo.org>:
>>>
>>> The timer keeps running if you set the dependency on the service.  So,
>>> next time the timer runs, it will try again.  You might want to just
>>> set an hourly job and have it check for a successful run in the last
>>> day or whatever.
>>>
> You could of course trigger this from either the mount or hourly.
> Anytime you mount the drive or every hour systemd will run the service,
> and the service will see if it managed to do a backup/etc in the last
> day/week/whatever, and then run if appropriate.

This is actually how I setup several former cron-jobs as systemd timers, 
here, based on an hourly check somewhat similar to what most crons 
(including gentoo's for over 10 years now and mandrake's before that) are 
actually setup to do to get around the fact that cron won't on-its-own 
trigger after restart if the machine was down or cron not running when 
the configured time for a job ran.

Here's how I have it setup here.  Note that my initials are jed, and I 
use them regularly as a prefix/suffix to denote custom configs (here, 
systemd units) I've created myself, as opposed to those shipped in 
whatever package.


/etc/systemd/system/jed.hourly.timer:

[Unit]
Description=JED Hourly timer

[Timer]
OnBootSec=10min
OnUnitActiveSec=1h
Unit=jed.hourly.target

[Install]
WantedBy=timers.target


timers.target is a systemd default target with the purpose of starting 
all the various timers.  The install entry says to install a symlink 
pointing to jed.hourly.timer in the timers.target.wants subdir, which 
will of course cause timers.target to start it.

The 10 minute OnBootSec setting says wait 10 minutes after boot before 
activating this timer the first time.  The 1h OnUnitActiveSec setting 
says reactivate it every hour after its first activation.  Thus, we have 
an hourly timer, except that its first run is offset from boot by only 10 
minutes, not a full hour.

Meanwhile, what it actually does when it runs is activate 
jed.hourly.target -- that's the Unit= line.


/etc/systemd/system/jed.hourly.target:

[Unit]
Description=JED Hourly target
StopWhenUnneeded=yes


This one, like most targets, is pretty simple, its only purpose being a 
synchronization point, activating a bunch of services and/or other units, 
generally those with symlinks installed in the corresponding 
jed.hourly.target.wants subdir, at once.

Note the StopWhenUnneeded=yes setting.  The sole purpose of activating 
this thing is to trigger all its wants deps to run, and as soon as it has 
done that, it can stop again.  This lets the target "rest" between hourly 
triggerings.


Symlinks in /etc/systemd/jed.hourly.target.wants/:

jed.hourly.timestamp.service


That's it, just the one.  With just it there it would have in fact been 
just as easy for me to set jed.hourly.timer's 
Unit=jed.hourly.timestamp.service, and omitted jed.hourly.target 
entirely.  However, using the target is MUCH more flexible, as it allows 
me to add new services I want triggered hourly to the same target, as I 
decide I want/need them.

If you're familiar with the usual cron setup, in particular, the usual 
/etc/crond.hourly/ (or whatever it was, that's from memory as I replaced 
my cronjobs with timers and don't even have a cron installed, these 
days), that's EXACTLY the function /etc/systemd/jed.hourly.target.wants 
ends up filling, with the minor exception being that as I set it up, it 
triggers hourly, starting 10 minutes after boot, instead of hourly, at 
some arbitrary minute of the hour.  That actually suits my purposes 
better than an arbitrary time of the hour would, but if people really 
wanted more-cron-line specific minutes of the hour, that would be set 
using OnCalendar= instead of the boot+10min and hourly thereafter, that I 
used.


/etc/systemd/jed.hourly.timestamp.service:

[Unit]
Description=JED Hourly timestamp updater

[Service]
Type=oneshot
ExecStart=/etc/systemd/local-scripts/jed.hourly.timestamp

[Install]
WantedBy=jed.hourly.target


This service simply runs the script named in execstart, oneshot, meaning 
it runs until its done, then stops.  Which is what we want, since we're 
triggering it once per hour via the timers above.

The install section simply tells systemd where to put that symlink 
mentioned above, in the appropriate target.wants subdir, when the service 
is "installed".


/etc/systemd/local-scripts/jed.hourly.timestamp:

#!/bin/bash

stampfile=/var/log/jed.hourly.day.stamp

# if stamp is old or doesn't exist, update stampfile and run...
doit(){
	touch $stampfile
	systemctl start jed.daily.target
	exit
}

[[ -f stampfile ]] || doit

yesterday=$(date -d yesterday +"%Y%m%d%H%M")

stamp=$(date -r "$stampfile" +"%Y%m%d%H%M")

[[ $stamp ]] || doit

[[ $stamp -lt $yesterday ]] && doit

exit 0


This is where the custom magic happens and the real work gets done.  If 
you're familiar with the usual cronjob runscripts setup, this will look 
similar.  Basically, systemd timers, etc, magic above only sets up an 
hourly systemd timer, not a daily, weekly, etc.  This script actually 
sets up a daily trigger, based on a daily timestamp that's checked every 
hour to see if it has been 24 hours or longer since the last run, updated 
if so, and the daily trigger triggered.

As I said, the idea is very similar to that used with a standard cronjob 
runscripts setup, since that's exactly where I got it from. =:^)  Like 
the cronjob setup, if the system was down the hour the daily would have 
otherwise triggered, this script ensures that it's triggered the next 
time the hourly timer is run... which will normally be 10 minutes after 
boot due to the 24 hours elapsing when the system was down, and the boot
+10min setting in that first unit, jed.hourly.timer, above.

OK, so what does this script trigger?  systemctl start jed.daily.target

Again, I /could/ have setup a daily timer much like the hourly timer, and 
handled it that way.  Instead, I chose to use the hourly time to activate 
a service to run this script, handling the daily mechanism that way.

(FWIW I put the stampfile itself in /var/log primarily because that's a 
conveniently writable place to put it.  My /, including much of /var, is 
mounted read-only by default, while /var/log obviously must be writable.  
And thought about from a particular viewpoint, that timestamp /is/ a log 
in some fashion, since it effectively logs the last time the daily jobs 
should have run.)

OK, we're almost there now. =:^)  But tying up loose ends...


/etc/systemd/system/jed.daily.target:

[Unit]
Description=JED Daily Target
StopWhenUnneeded=yes


Pretty much identical to the hourly target as the functionality is the 
same, just with a different name, reflecting the different timing.


Symlinks in /etc/systemd/system/jed.daily.target.wants:

jed.daily.logrotate.service.


For the moment that's it.  I haven't needed weekly, monthly, etc, 
scheduling, so I've not created it.  But having done the hourly to daily 
thing, I could easily copy and convert that to weekly or whatever, 
triggered by the daily, just as daily is triggered by the hourly.

And to tie up the final loose end before I apply the above to the case of 
this thread, as well as to supply a sample logrotate systemd service for 
anyone else wanting to switch to systemd timer based triggering and get 
rid of cron's triggering (while noting that the systemd way to do it 
would be to use tmpfiles.d settings, but I still find logrotate useful 
for doing actual log rotation, thus keeping those settings separate from 
systemd's config)...


/etc/systemd/system/jed.daily.logrotate.service:

[Unit]
Description=JED daily logrotate

[Service]
Type=oneshot
Nice=10
ExecStart=/usr/sbin/logrotate --state /var/log/logrotate.state /etc/
logrotate.conf

[Install]
WantedBy=jed.daily.target


(Note the wrap of the execstart line...)

###################

OK, so how does all that apply to the case of this thread? =:^)

Simple enough.  Where I threw in that jed.hourly.timestamp.service systemd 
unit file (that is, symlinked into the hourly target.wants subdir) and 
jed.hourly.timestamp script that it called, you'd throw in another 
service, either in addition to the timestamp one I used, or if you don't 
want to use my daily solution, in place of it.


The key thing to understand here is that once you've setup a service to 
run a script you create yourself, you can have it do literally whatever 
you want it too, just as you would with any other script -- it's NO 
LONGER limited to the strict declarative style and options systemd 
provides -- you're ONLY using systemd to /start/ it. =:^)

So... suppose you want it to check hourly to see if it can and should 
trigger something else, but only run once a day... the timer and 
timestamp-script-based framework I have above already does the hourly 
check part, as well as the only running once a day part.

If you prefer that it check every 10 minutes, it's simple enough to 
change that hourly thing to 10 minutes.  If you like it hourly but want 
it to ALWAYS be hourly, including not checking the first time until an 
hour after boot, that's simple enough as well.  Similarly, if you want 
the final action to run only every two days, or twice a day, or 
whatever.  It's simple enough to change the timestamp mechanism to make 
that happen.

Meanwhile, with the basic framework already there, it should also be 
simple enough to plugin some conditional logic testing to see if the 
partition device-files are showing up or not.  If not, simply quit 
without updating the timestamp and wait for the next hourly or whatever 
check to roll around.  If so, and the timestamp is also expired, then run 
the scripted backup or whatever else you were wanting it to do.

... And of course you can rename them from jed.* to your own name of 
choice, too.  Jed.* is fine for my own usage, but it would look a bit 
ridiculous, and could well defeat its site-specific-id purpose if someone 
actually decided to ship it as part of some package, if that jed.* naming 
pattern remained as installed out of my own control.  But be sure to 
change the names /in/ the files too, not just the names /of/ the files, 
or systemd will be complaining about the stupid admin trying to give it 
nonsensical unit files to run. =:^P

Also, one last caution.  I chose to retype most of the above files 
manually, getting back into what they did and how as I did so, instead of 
copy/pasting them and then sitting there studying them to remember how 
they worked.  Not so boring that way, but there's possibly a few typos I 
missed.  They should be reasonably obvious and easy to fix, tho, if I did 
miss some...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 2572 bytes --]

Am Mon, 2 Mar 2015 05:13:26 +0000 (UTC)
schrieb Duncan <1i5t5.duncan@cox.net>:

> Rich Freeman posted on Sun, 01 Mar 2015 14:13:53 -0500 as excerpted:
> 
> > On Sun, Mar 1, 2015 at 1:20 PM, Marc Joliet <marcec@gmx.de> wrote:
> >>
> >> Regardless: thoughts?
> > 
> > I'd probably just do this:
> >> Am Sun, 1 Mar 2015 08:34:19 -0500 schrieb Rich Freeman
> >> <rich0@gentoo.org>:
> >>>
> >>> The timer keeps running if you set the dependency on the service.  So,
> >>> next time the timer runs, it will try again.  You might want to just
> >>> set an hourly job and have it check for a successful run in the last
> >>> day or whatever.
> >>>
> > You could of course trigger this from either the mount or hourly.
> > Anytime you mount the drive or every hour systemd will run the service,
> > and the service will see if it managed to do a backup/etc in the last
> > day/week/whatever, and then run if appropriate.
> 
> This is actually how I setup several former cron-jobs as systemd timers, 
> here, based on an hourly check somewhat similar to what most crons 
> (including gentoo's for over 10 years now and mandrake's before that) are 
> actually setup to do to get around the fact that cron won't on-its-own 
> trigger after restart if the machine was down or cron not running when 
> the configured time for a job ran.
> 
> Here's how I have it setup here.  Note that my initials are jed, and I 
> use them regularly as a prefix/suffix to denote custom configs (here, 
> systemd units) I've created myself, as opposed to those shipped in 
> whatever package.
> 
[GIGASNIP thorough explanation ;) ]

I pretty much thought that's what Rich was alluding to, but thanks for showing
that it's not *that* much extra complication (and how one can use a target unit
for this). I never really looked at how these things are done by run-crons (and
similar).

Just for completeness: I use fcron instead of vixie-cron, so some of the stuff
systemd timers can do was already known to me.   For example, in fcron, lines
can start with "@" to denote that they run relative to system startup (e.g, "@
5" for "every five minutes after boot).  The "first" option specifies how long
to wait before starting an entry for the first time, analogous to "OnBootSec".

Anyway, like I mentioned before, I'll revisit this once I've solved the HDD
problem (or not, if it turns out to be a firmware issue).

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Rich Freeman posted on Tue, 24 Feb 2015 16:44:59 -0500 as excerpted:

> On Tue, Feb 24, 2015 at 3:15 PM, Marc Joliet <marcec@gmx.de> wrote:
>>
>> == Things I have *not* gotten rid of (yet) ==
>>
>> I plan on uninstalling syslog-ng, but haven't done so yet.  I simply
>> feel better waiting a bit, even though I don't run it anymore.  Man, I
>> feel silly after typing that...
> 
> Yeah, took me a while to uninstall it, but I don't find much value in
> running both.

First, welcome to systemd! =:^)  I may respond to other parts too but 
this one's reasonably easy, while my choice is different, so I'll explain 
what and why...

Intro: I chose to keep syslog-ng running here, while I (of course) have 
journald running as well.  I believe the way I have configured both plays 
to the advantages of each, while discounting the weak areas of each as 
the other one covers that area.

What:

1) Journald is configured to use volatile storage (the /run/log/journal 
subdir, with /run being tmpfs) ONLY, effectively limiting it to single-
session.

2) Syslog-ng is configured pretty much as before, except that I have its 
systemd USE flag turned on, which builds it with journald support so the 
two cooperate a bit better.

Without looking too much into details, I believe USE=systemd for syslog-ng 
tells it to build against a library systemd/journald provides for that 
purpose, such that syslog-ng's "system" source listens to journald 
instead of /dev/log and the kernel (dmesg) directly.

What I can say is that while I've seen complaints from others about 
loggers getting duplicate entries (from systemd and the kernel both, for 
instance) in some cases, or missing entries in others (the OP, here), 
I've not noticed anything like that, here, and I believe it's due to the 
above -- only having the "system" and internal syslog-ng sources 
configured so syslog-ng isn't getting the same message from two sources, 
and building with the systemd USE flag, which basically configures syslog-
ng's system source to be lazy and let journald feed it, instead of doing 
all the work of fetching and parsing the kernel and syslog socket on its 
own.

Why:

I love the journald/systemd integration -- having the last few log 
messages on systemctl status <service> makes troubleshooting within the 
same session **MUCH** easier, and the other information journald keeps is 
also useful.  But, for me at least, that's most useful in the same 
session.  Once I reboot and the information I'm looking for is in a 
previous session, the usefulness of that extra information goes down 
quite a bit.

Meanwhile, for logs beyond one session, I still prefer traditional text-
based logs, tho I suppose that's partly because that's what I'm most used 
to.  But there's several advantages to it that I see:

a) If the system crashes, partially corrupted in-the-crash text logs can 
be of at least some use after a reboot.  Binary journals, not so much.

b) As far as I could find, journald has absolutely no mechanism to filter 
incoming (pre-storage) log entries, while syslog-ng has had (pre-storage) 
filter mechanisms for ages.  Journald seems to have all sorts of outgoing 
post-storage filter-on-output options, some of which are pretty nifty, 
but there have been several times in the past where logs filled up with 
"noise", thousands of repeat entries for something or other that occurs 
regularly, say once a minute, that I simply do not care about and don't 
want in my logs at all, but which I can't directly stop at the source for 
whatever reason.  One of them was a kernel bug that triggered for a few 
kernel cycles, then eventually went away, another was a ksysguard 
triggered message, with that ksysguard graph updating every second, etc.  
Unfortunately, while journald makes it easy to filter these out on 
output, I found no way of preventing their appearance in the journal at 
all.

But syslog-ng lets me dump them without ever actually logging them, or 
route them to a different log file if I prefer, keeping my primary logs 
clean. =:^)

As a counter-point, however, at least journald does compression to some 
degree, so tens of thousands of identical or nearly identical messages 
don't take up as much room as you might expect as they compress very 
well. =:^)  I guess after the first instance, dups basically consist of a 
timestamp and a pointer to the first message.  So they don't take much 
space.  Which is fortunate when some "noise" message starts appearing 
once a second... and you're storing it in tmpfs, thus in memory!  
Obviously persistent logs would be similarly space efficient, but I still 
don't like the idea of not being able to filter out the noise, BEFORE it 
hits permanent storage, so I just make sure it /doesn't/ hit permanent 
storage. =:^)

c) I use btrfs for my primary filesystems, and btrfs and journald's 
binary-format journals don't play so well together.  FWIW, the latest 
systemd-219 has lots of improvements for btrfs users, including some new 
features that are btrfs-specific (which I never-the-less don't expect to 
use right away as they're mostly VM/container oriented and I don't do 
much of that sort of thing, but I may well eventually use them), as well 
as some changes that should make journaling on btrfs work somewhat 
better.  So this point should be to some degree eliminated after that 
stuff stabilizes tho part of the fix is journald setting nocow on the 
journal files and that has other implications I'll skip further 
discussion of here.  However, by limiting journald to tmpfs only, I still 
get the best benefits of systemd/journald integration (like status 
including the last few log messages as mentioned above), while 
eliminating the entire class of btrfs/journald issues.  Best of both 
worlds. =:^)

d) Tho I don't do anything with it in my syslog-ng config here, syslog-ng 
can actually make use of some of the extra information journald passes to 
it, including trusted per-app filtering and routing.  One of the problems 
that the old syslog approach had was that anything that could log 
messages could claim to be anything it wanted, and syslog really couldn't 
tell whether it was telling the truth or not, all it could do is sort and 
report the messages based on what was claimed.  A big bullet-point 
feature of systemd/journald is that because systemd enforces service 
separation via control-classes and the like, there's a much stronger 
guarantee that what systemd/journald says is service X, is REALLY service 
X and not service Y!  But that's not limited to journald.  Syslog-ng has 
for several versions now had the ability to filter, route and report on 
out-of-log-channel information such as the extra properties that journald 
associates with each message, that make features such as systemctl status 
reporting the last few log messages from that service possible, because 
systemd/journald actually tracks it, AND passes it on to other loggers as 
well now, providing they're built to use the API provided by the systemd 
library I mentioned above.  And again, for gentooers, syslog-ng is built 
with that API, against that library, based on USE=systemd.  So once 
that's there, if desired, you can do all sorts of fancy filtering/log-
routing/etc based on information such as the application/service name, 
that systemd passes along to syslog-ng.

And again, with syslog-ng, this can be done BEFORE the actual logging if 
desired, instead of logging everything and filtering only on output, as 
journald does. =:^)

e) I'm simply more comfortable dealing with text-based logs, since I'm 
used to them and can use whatever tools I want on them, as well as being 
able to split/sort/route to multiple log files in as simple or complex a 
setup as I want. =:^)

Summary:

For me, splitting up logging duties so journald does the collection, but 
only does session journaling to tmpfs, while syslog-ng still handles the 
"persistent" logging, gives me the best of both worlds.  I love systemd/
journald features such as log-messages-in-status and having seen how 
useful it is, would seriously find it hard to do without.  But at the 
same time, I can't bring myself to accept persistent logging without the 
ability to filter out the noise before it hits persistent storage, and 
journal storage on btrfs is problematic anyway, with both problems nicely 
solved by handing off to syslog-ng for my persistent storage needs while 
keeping journald for same-session use, so that's exactly what I do. =:^)


-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Rich Freeman]

On Wed, Feb 25, 2015 at 5:13 AM, Duncan <1i5t5.duncan@cox.net> wrote:
>
> a) If the system crashes, partially corrupted in-the-crash text logs can
> be of at least some use after a reboot.  Binary journals, not so much.

Have you tested this, or found some other data to support this?  I
think that journalctl does parse truncated files.

>
> But syslog-ng lets me dump them without ever actually logging them, or
> route them to a different log file if I prefer, keeping my primary logs
> clean. =:^)

I was thinking about this and another advantage of split log files is
that you could have different rotation/retention policies for each.  I
believe journald's log rotation is one-size-fits-all.

-- 
Rich

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Rich Freeman posted on Wed, 25 Feb 2015 07:13:03 -0500 as excerpted:

> On Wed, Feb 25, 2015 at 5:13 AM, Duncan <1i5t5.duncan@cox.net> wrote:
>>
>> a) If the system crashes, partially corrupted in-the-crash text logs
>> can be of at least some use after a reboot.  Binary journals, not so
>> much.
> 
> Have you tested this, or found some other data to support this?  I think
> that journalctl does parse truncated files.

I believe I've seen posts to that effect on the btrfs list (my biggest 
non-gentoo participation).  Note that journald does rotation of I believe 
8 journal files by default, and ideally, it'd only be the last one chewed 
up.  Also, cleanly truncated is one thing; file-system-corrupted so it 
ends up with a piece of some other totally different file in it (as most 
more modern filesystems take pains to prevent, at least on Linux, as the 
other file might be from another user and that ends up being a security 
risk, so modern filesystems generally zero-truncate it if they think it 
might be corrupted in that manner) is something entirely different.

However, fair point.  My support on this bit /is/ a bit fuzzier than I'd 
like.  Thanks for pointing that out, to me as well.  =:^)

>> But syslog-ng lets me dump them without ever actually logging them, or
>> route them to a different log file if I prefer, keeping my primary logs
>> clean. =:^)
> 
> I was thinking about this and another advantage of split log files is
> that you could have different rotation/retention policies for each.  I
> believe journald's log rotation is one-size-fits-all.

I believe so... because everything journald logs (well, everything 
system, user session logs are I'm pretty sure treated separately, at 
least with persistent storage enabled, there's a note somewhere in the 
documentation that if volatile-only is set, journald throws user journals 
in with the system journals too, which implies it doesn't, with 
persistent storage) is thrown in the same log, so there's little choice 
on rotation strategy.

But of course syslog-ng allows you to split the logs however you want, 
and route individual log messages to one or more logs.  And those logs in 
turn can be rotated on entirely different policies.

Thanks for that point too.  I'm so used to thinking in terms of being 
able to split logs and setup individual logrotate policies for them, I 
hadn't even considered that implication of the common journald journals.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 9690 bytes --]

Am Wed, 25 Feb 2015 10:13:14 +0000 (UTC)
schrieb Duncan <1i5t5.duncan@cox.net>:

> Rich Freeman posted on Tue, 24 Feb 2015 16:44:59 -0500 as excerpted:
> 
> > On Tue, Feb 24, 2015 at 3:15 PM, Marc Joliet <marcec@gmx.de> wrote:
> >>
> >> == Things I have *not* gotten rid of (yet) ==
> >>
> >> I plan on uninstalling syslog-ng, but haven't done so yet.  I simply
> >> feel better waiting a bit, even though I don't run it anymore.  Man, I
> >> feel silly after typing that...
> > 
> > Yeah, took me a while to uninstall it, but I don't find much value in
> > running both.
> 
> First, welcome to systemd! =:^)  I may respond to other parts too but 
> this one's reasonably easy, while my choice is different, so I'll explain 
> what and why...
> 
> Intro: I chose to keep syslog-ng running here, while I (of course) have 
> journald running as well.  I believe the way I have configured both plays 
> to the advantages of each, while discounting the weak areas of each as 
> the other one covers that area.
> 
> What:
> 
> 1) Journald is configured to use volatile storage (the /run/log/journal 
> subdir, with /run being tmpfs) ONLY, effectively limiting it to single-
> session.

Right, I remember that from your thread a while back.

> 2) Syslog-ng is configured pretty much as before, except that I have its 
> systemd USE flag turned on, which builds it with journald support so the 
> two cooperate a bit better.
[...]
> Meanwhile, for logs beyond one session, I still prefer traditional text-
> based logs, tho I suppose that's partly because that's what I'm most used 
> to.  But there's several advantages to it that I see:

Personally, I don't mind the binary format, since the engineering advantages
convinced me (constant-time access, lots of meta-data which you can filter by,
git-inspired chain of hashes for integrity checking, etc.).

> a) If the system crashes, partially corrupted in-the-crash text logs can 
> be of at least some use after a reboot.  Binary journals, not so much.

As Rich mentioned, and as I remember reading myself (on the btrfs ML?), the
journal will do its best to read back from corrupted logs and is supposedly
pretty resilient.

> b) As far as I could find, journald has absolutely no mechanism to filter 
> incoming (pre-storage) log entries, while syslog-ng has had (pre-storage) 
> filter mechanisms for ages.  Journald seems to have all sorts of outgoing 
> post-storage filter-on-output options, some of which are pretty nifty, 
> but there have been several times in the past where logs filled up with 
> "noise", thousands of repeat entries for something or other that occurs 
> regularly, say once a minute, that I simply do not care about and don't 
> want in my logs at all, but which I can't directly stop at the source for 
> whatever reason.  One of them was a kernel bug that triggered for a few 
> kernel cycles, then eventually went away, another was a ksysguard 
> triggered message, with that ksysguard graph updating every second, etc.  
> Unfortunately, while journald makes it easy to filter these out on 
> output, I found no way of preventing their appearance in the journal at 
> all.
> 
> But syslog-ng lets me dump them without ever actually logging them, or 
> route them to a different log file if I prefer, keeping my primary logs 
> clean. =:^)

That is a very good point, I hadn't thought of that.

However, I think that I prefer the post-filtering approach, since it is in
principle the more flexible one.  At work I sometimes mix the logs from
different units (e.g., units A and B, and then units B, C, and D).

But I realised today that journalctl has no negation operator!  There is,
however, an RFE for it.

But regardless of what you use, I think that the worst offenders are services
that write logs themselves (I'm looking at you, samba).

> As a counter-point, however, at least journald does compression to some 
> degree, so tens of thousands of identical or nearly identical messages 
> don't take up as much room as you might expect as they compress very 
> well. =:^)  I guess after the first instance, dups basically consist of a 
> timestamp and a pointer to the first message.  So they don't take much 
> space.  Which is fortunate when some "noise" message starts appearing 
> once a second... and you're storing it in tmpfs, thus in memory!  
> Obviously persistent logs would be similarly space efficient, but I still 
> don't like the idea of not being able to filter out the noise, BEFORE it 
> hits permanent storage, so I just make sure it /doesn't/ hit permanent 
> storage. =:^)

Yeah, I think that's a good use case for pre-filtering: an overly noisy service
whose logs you purposely want to ignore (for example, because you're still
testing it - but then again, for that particular case you could probably use
systemd-nspawn and log to an isolated journal).

> c) I use btrfs for my primary filesystems, and btrfs and journald's 
> binary-format journals don't play so well together.  FWIW, the latest 
> systemd-219 has lots of improvements for btrfs users, including some new 
> features that are btrfs-specific (which I never-the-less don't expect to 
> use right away as they're mostly VM/container oriented and I don't do 
> much of that sort of thing, but I may well eventually use them), as well 
> as some changes that should make journaling on btrfs work somewhat 
> better.  So this point should be to some degree eliminated after that 
> stuff stabilizes tho part of the fix is journald setting nocow on the 
> journal files and that has other implications I'll skip further 
> discussion of here.  However, by limiting journald to tmpfs only, I still 
> get the best benefits of systemd/journald integration (like status 
> including the last few log messages as mentioned above), while 
> eliminating the entire class of btrfs/journald issues.  Best of both 
> worlds. =:^)

Well, I'm on an SSD, but even on the laptop I haven't noticed any performance
issues (yet).  Then again, I use autodefrag, so that probably helps.

What's funny though is that the systemd news file
(http://cgit.freedesktop.org/systemd/systemd/tree/NEWS) occasionally
refers to non-btrfs file systems as "legacy file sysetms".  At least, as a
btrfs user I think it's funny :) .

> d) Tho I don't do anything with it in my syslog-ng config here, syslog-ng 
> can actually make use of some of the extra information journald passes to 
> it, including trusted per-app filtering and routing.  One of the problems 
> that the old syslog approach had was that anything that could log 
> messages could claim to be anything it wanted, and syslog really couldn't 
> tell whether it was telling the truth or not, all it could do is sort and 
> report the messages based on what was claimed.  A big bullet-point 
> feature of systemd/journald is that because systemd enforces service 
> separation via control-classes and the like, there's a much stronger 
> guarantee that what systemd/journald says is service X, is REALLY service 
> X and not service Y!  But that's not limited to journald.  Syslog-ng has 
> for several versions now had the ability to filter, route and report on 
> out-of-log-channel information such as the extra properties that journald 
> associates with each message, that make features such as systemctl status 
> reporting the last few log messages from that service possible, because 
> systemd/journald actually tracks it, AND passes it on to other loggers as 
> well now, providing they're built to use the API provided by the systemd 
> library I mentioned above.  And again, for gentooers, syslog-ng is built 
> with that API, against that library, based on USE=systemd.  So once 
> that's there, if desired, you can do all sorts of fancy filtering/log-
> routing/etc based on information such as the application/service name, 
> that systemd passes along to syslog-ng.
> 
> And again, with syslog-ng, this can be done BEFORE the actual logging if 
> desired, instead of logging everything and filtering only on output, as 
> journald does. =:^)

That's actually pretty cool :) . You're basically using the journal as a log
accumulator and syslog-ng for routing and post-processing its messages.

Whiles that's not how I want to do things, I still appreciate how cool that
is :) (and the fact that the journal can be used that way).

> e) I'm simply more comfortable dealing with text-based logs, since I'm 
> used to them and can use whatever tools I want on them, as well as being 
> able to split/sort/route to multiple log files in as simple or complex a 
> setup as I want. =:^)
> 
> Summary:
> 
> For me, splitting up logging duties so journald does the collection, but 
> only does session journaling to tmpfs, while syslog-ng still handles the 
> "persistent" logging, gives me the best of both worlds.  I love systemd/
> journald features such as log-messages-in-status and having seen how 
> useful it is, would seriously find it hard to do without.  But at the 
> same time, I can't bring myself to accept persistent logging without the 
> ability to filter out the noise before it hits persistent storage, and 
> journal storage on btrfs is problematic anyway, with both problems nicely 
> solved by handing off to syslog-ng for my persistent storage needs while 
> keeping journald for same-session use, so that's exactly what I do. =:^)

Right :) .

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Marc Joliet posted on Wed, 25 Feb 2015 19:56:32 +0100 as excerpted:

> But regardless of what you use, I think that the worst offenders are
> services that write logs themselves (I'm looking at you, samba).

>> c) I use btrfs for my primary filesystems, and btrfs and journald's
>> binary-format journals don't play so well together. [...]
> 
> Well, I'm on an SSD, but even on the laptop I haven't noticed any
> performance issues (yet).  Then again, I use autodefrag, so that
> probably helps.

Autodefrag does help.

There are two related issues at work, here.

The primary one is that pretty much any COW-based filesystem, including 
btrfs, is going to have problems with internal-rewrite-pattern (as 
opposed to append-only rewrites) files of any significant size.  At the 
small end this includes sqlite database files such as those firefox and 
other mozilla products use.  These, autodefrag manages well.

At the larger end are multi-gig VM images and similarly sized database 
files.  These, autodefrag doesn't manage so well, particularly if writes 
are coming in at any significant rate, because at some point it's going 
to take longer to rewrite the entire file (or even the affected normally 
one-gig data chunk) than the time between incoming writes.

And the place where such fragmentation REALLY shows up is trying to run 
btrfs filesystem maintenance commands like balance.  On a sufficiently 
fragmented filesystemsystem, particularly with quotas on too as their 
tracking significantly complicates things, balance can take WEEKS on a 
single-digits terabyte filesystem.

IOW, a lot of people don't notice it until something goes wrong and 
they're trying to replace a failed device with one of the btrfs raid 
modes, etc.  That's a nasty time to find out how tangled things were, and 
realize it'll take weeks to sort out, during which another device could 
well fail, leaving you high and dry!

The immediate (partial) solution to the problem with these large files, 
typically over a gig, is to set them nocow (which on btrfs must be done 
at creation time, while the file is still zero-sized, in ordered to take 
proper effect; this is normally accomplished by setting the directory 
they'll be in to nocow, which doesn't affect the directory itself, but 
does cause any newly created files or subdirs in it to inherit the nocow 
attribute).

And this is actually what systemd-219 is doing with the journal files now.

But, setting nocow automatically disables both transparent compression 
(if otherwise enabled) and checksumming.  The latter isn't actually as 
bad as one might expect, because most applications (including systemd/
journald) that deal with such files already have some sort of builtin 
corruption detection and possible repair functionality -- they have to in 
ordered to work acceptably on traditional filesystems that didn't do 
filesystem level checksumming, and letting them have at it would indeed 
seem to be the best policy in this case.

The second, related problem, is snapshotting.  Because snapshotting 
relies on COW, snapshotting a nocow file forces it to effectively cow-1 
-- the first time a block is rewritten after a snapshot, it is cowed, 
despite the ordinary nocow.  Now setup say hourly auto-snapshotting using 
snapper or the like, and continue to write to that "nocow" file, and 
pretty soon it'll be as fragmented as if it weren't nocow at all!

With careful planning, separate subvolumes for the nocow files so they 
aren't snapshotted with the rest of the system, snapshotting the nocow 
subvolume with a period near the low frequency end of your target range 
(say every other day or weekly instead of daily or twice a day), and if 
they aren't rotated out regularly, periodic scripted btrfs defrags (say 
weekly or monthly) of the affected files, good admins generally can keep 
fragmentation from this source at least within reason.

And systemd-219 is actually creating a separate subvolume for its journal 
files now, by default, thus keeping them out of the general system (or 
/var) snapshot.  But while both that and nocowing the journal files now 
does help, it's still a reasonably fragile solution, as long as admins 
don't realize what's going on, and can be tempted to set daily or more 
frequent snapshotting on the journal subvolume too (or if the subvolume 
doesn't take, say because it's an existing installation where there's 
already a directory by that name and thus there can't be a subvolume at 
the same place with the same name).


**BUT A BIG CAVEAT** lest anyone on stable with btrfs and systemd jump 
onto 219 too fast.  Yes, 219 DOES have some nice new features.  
Unfortunately, it's broken in a few new ways as well.

* Apparently, systemd-219's networkd breaks with at least static IPv4-
only configurations, as my network failed to come up with it.  From the 
errors it was trying IPv6 and because that failed (it's not even in my 
kernel), it gave up and didn't even try IPv4, instead trying to set the 
IPv4 IP and gateway values into IPv6, which obviously isn't going to work 
at all!

* There's also issues with the new tmpfiles.d configuration that has 
replaced d lines (create a directory if it doesn't exist) with new v 
lines (create a subvolume if on btrfs and possible, else fallback to d 
behavior and create a directory), because subvolume creation fails 
differently than directory creation, and the differences aren't all 
sorted, yet.

Hopefully, systemd-220 will fix the IPv4 issue and bring a bit more 
maturity to the tmpfiles.d subvolumes-creation feature by properly 
falling back to d/directories if need be, instead of erroring out.  
Meanwhile, hopefully a gentoo systemd-219-rX release will fix some of 
these issues as well.  But for right now, I'd suggest staying away from 
it, as it's definitely not prime-time ready in its current form.

FWIW, I'm back on 218-r3 for now, done with a quick emerge --pkgonly 
<systemd-219.  I've not yet masked 219, however, so an update will try to 
bring it back in, and I will thus have to see what changes have happened 
and either mask it or try building it again, next time I update.

> What's funny though is that the systemd news file
> (http://cgit.freedesktop.org/systemd/systemd/tree/NEWS) occasionally
> refers to non-btrfs file systems as "legacy file sysetms".  At least, as
> a btrfs user I think it's funny :) .

Indeed.  They've definitely adopted btrfs and are running with it.  If 
you've read anything about their plans, the features of btrfs really do 
provide a filesystem-side ready-made solution for them to adopt, altho 
I'd still not call btrfs itself exactly mature -- even more than with 
other filesystems, if an admin is putting data on btrfs and doesn't have 
tested backups available, they really do NOT value that data, claims to 
the contrary not withstanding.

And in a way, it's good, because systemd pushing it like that means 
systemd based distros will be pushing it too, which will bring far wider 
deployment of btrfs, ready or not, which will in turn help btrfs mature 
faster with all those additional strange-corner-case bug reports and 
hopefully fixes.  I just feel for the poor admins trusting their distro 
as they head into this without the backups they really should have... as 
ultimately, a lot of them are unfortunately going to have to learn that 
no backups really DOES mean you'd rather lose that data than bother with 
backups, lesson, the HARD way! =:^(

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Marc Joliet posted on Tue, 24 Feb 2015 21:15:45 +0100 as excerpted:

> I uninstalled acpid, since the Gentoo Wiki page on systemd mentions that
> it might not be necessary (see the question below).  Being able to do
> this was another reason I switched to tlp, since it doesn't depend on
> acpid.  So far I have yet to notice anything amiss.

FWIW, I still have acpid installed here, and hadn't really thought about 
it.  Thanks for the heads-up.  I'll have to investigate...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Marc Joliet posted on Tue, 24 Feb 2015 21:15:45 +0100 as excerpted:

> I would like to convert some programs I start in .xprofile to units that
> are started by my users's systemd instance.  I started off with mpd,

It looks like you got this sorted, but FWIW...

I just use the standard system level mpd.service here.  It drops privs 
and runs as its own user, and it listens localhost for connections from 
clients run as normal users, which pretty well limits any potential 
security issues, and that means I can control it via either my normal X 
user and GUI client, or mpc or whatever as either my normal user or my 
admin user (or root, for that matter).

In fact, IMO that's mpd's biggest advantage over most other players.  I 
don't have to be in X to play music, and in fact, I can not only quit X, 
but log out as the user I started playing the music with, and have it 
continue uninterrupted, controlling it when I need to using mpc from 
whatever other user (here, normally my admin user).

IOW, I really do use it as the system service/daemon that the name, mpd, 
music player daemon, implies. 

And as such, I don't /want/ to start it from user login or service, I 
want it as a system service, which is how I run it. =:^)

But you're king of your own boxes.  If you want to run it as a user-level 
service and have it quit when you logout that user, go right ahead.  
Don't let the fact that such usage bothers me and I don't see the point, 
if you're running it as your normal user anyway, there's all those other 
player apps, why use mpd in that case, bother you.  (No, that isn't 
sarcasm.  Yes, it does bother me, but it's your box and your choice, and 
presuming to take that away from you would bother me **FAR** more.  So do 
it the way that works best for you! =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 3893 bytes --]

Am Wed, 25 Feb 2015 10:33:37 +0000 (UTC)
schrieb Duncan <1i5t5.duncan@cox.net>:

> Marc Joliet posted on Tue, 24 Feb 2015 21:15:45 +0100 as excerpted:
> 
> > I would like to convert some programs I start in .xprofile to units that
> > are started by my users's systemd instance.  I started off with mpd,
> 
> It looks like you got this sorted, but FWIW...
> 
> I just use the standard system level mpd.service here.  It drops privs 
> and runs as its own user, and it listens localhost for connections from 
> clients run as normal users, which pretty well limits any potential 
> security issues, and that means I can control it via either my normal X 
> user and GUI client, or mpc or whatever as either my normal user or my 
> admin user (or root, for that matter).
> 
> In fact, IMO that's mpd's biggest advantage over most other players.  I 
> don't have to be in X to play music, and in fact, I can not only quit X, 
> but log out as the user I started playing the music with, and have it 
> continue uninterrupted, controlling it when I need to using mpc from 
> whatever other user (here, normally my admin user).
> 
> IOW, I really do use it as the system service/daemon that the name, mpd, 
> music player daemon, implies. 
> 
> And as such, I don't /want/ to start it from user login or service, I 
> want it as a system service, which is how I run it. =:^)

I used to run it like that when I was still using pure ALSA, but then I
switched to pulseaudio, and didn't want to be in the unsupported "running
pulseaudio in system mode" class of users.  I actually did run mpd as a system
service that connected to my users pulseaudio instance, but that was... such an
ugly hack.

But either way, if you start mpd in ~/.xprofile, for example, and let it
daemonise, it will still keep running after you log out.  And mpd still doesn't
exit when I quit my desktop session, so that has yet to be a problem (more
precisely, the user instance of systemd doesn't exit).

But if you want to just turn your computer on and listen to music, then that's
obviously suboptimal, having to log in to start your music player ;) .

(Personally, I want to set up a NAS and a RaspberryPi or similar as a sort of
music center (i.e., I don't plan on archiving DVD rips or anything, at least
not currently).  My desktop would be too obtrusive and power hungry for such a
setup.)

> But you're king of your own boxes.  If you want to run it as a user-level 
> service and have it quit when you logout that user, go right ahead.  
> Don't let the fact that such usage bothers me and I don't see the point, 
> if you're running it as your normal user anyway, there's all those other 
> player apps, why use mpd in that case, bother you.  (No, that isn't 
> sarcasm.  Yes, it does bother me, but it's your box and your choice, and 
> presuming to take that away from you would bother me **FAR** more.  So do 
> it the way that works best for you! =:^)

Again, it *doesn't* terminate when I log out.  Secondly, I like the various UIs
for it (I primarily use mpc with the media keys of my keyboard, and ncmpcpp and
sonata otherwise, though mostly the former), and that I can control it remotely
(I do so from the laptop often).

But really, what I like the most is the background nature of it.  Or put
differently, mpd is unobtrusive (I mean, it has to be, it's a daemon for crying
out loud ;) ).  I can start playing music and forget that MPD is there.  The
music is simply "there", if you get what I mean.

On the laptop, however, I do in fact run something else when I'm not at home,
namely MOC, which is the next best thing (it can daemonise, too, and is
similarly unobtrusive, but only has one UI).

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Rich Freeman]

On Wed, Feb 25, 2015 at 2:17 PM, Marc Joliet <marcec@gmx.de> wrote:
> Am Wed, 25 Feb 2015 10:33:37 +0000 (UTC)
> schrieb Duncan <1i5t5.duncan@cox.net>:
>
>> But you're king of your own boxes.  If you want to run it as a user-level
>> service and have it quit when you logout that user, go right ahead.
>
> Again, it *doesn't* terminate when I log out.

FYI - this behavior is completely configurable - you can enable or
disable linger for any particular user.  I believe you can also use
this to do things like kill screen sessions left behind by a user.  I
think the default is for systemd to not clean up when a user's session
ends, but systemd DOES cleanup with a vengeance when a service ends
(so if you're like me and spawn background processes from shell
scripts in something like cron.daily then make sure you wait somewhere
so that systemd doesn't kill everything when your timer script reaches
the end - or you can tell systemd to not do cleanup but I don't think
that is a great practice).


-- 
Rich

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 1846 bytes --]

Am Wed, 25 Feb 2015 14:31:55 -0500
schrieb Rich Freeman <rich0@gentoo.org>:

> On Wed, Feb 25, 2015 at 2:17 PM, Marc Joliet <marcec@gmx.de> wrote:
> > Am Wed, 25 Feb 2015 10:33:37 +0000 (UTC)
> > schrieb Duncan <1i5t5.duncan@cox.net>:
> >
> >> But you're king of your own boxes.  If you want to run it as a user-level
> >> service and have it quit when you logout that user, go right ahead.
> >
> > Again, it *doesn't* terminate when I log out.
> 
> FYI - this behavior is completely configurable - you can enable or
> disable linger for any particular user.

Ah, then the *-linger commands to loginctl are related to this?  However,
loginctl says:

    % loginctl show-user 1000
    UID=1000
    GID=100
    Name=marcec
    Timestamp=Mi 2015-02-25 18:36:59 CET
    TimestampMonotonic=11724536
    RuntimePath=/run/user/1000
    Service=user@1000.service
    Slice=user-1000.slice
    State=active
    IdleHint=no
    IdleSinceHint=0
    IdleSinceHintMonotonic=0
    Linger=no

Hmm, one more thing to look this up, I guess.

Ah, I think I found it: I think it's the KillUserProcesses option
in logind.conf(5), which defaults to "no";

    KillUserProcesses=
        Takes a boolean argument. Configures whether the processes of a user
        should be killed when the user completely logs out (i.e. after the
        user's last session ended). Defaults to "no".

Perhaps I'll explicitly configure that, just so an upgrade doesn't accidentally
break things.

Ah, and looking at loginctl(1) now I understand what linger means: it lets you
start systemd user sessions at boot, without having to log in (so I was wrong
in the MPD sub-thread).  Nice!

[...]

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 8975 bytes --]

Hi,

it's been a while, but there are a few things to report.  So here it goes:

Am Tue, 24 Feb 2015 21:15:45 +0100
schrieb Marc Joliet <marcec@gmx.de>:

[...]
> == Network migration on my desktop ==
> 
> My desktop has a somewhat more complicated network setup than the laptop (which
> uses NetworkManager).  The wan0 network device is no problem, but I also have a
> bridge with one physical device connected to it, and one dummy device.
> 
> Thus, I had to migrate my netifrc based configuration.  Due to word of mouth, I
> decided on netctl.  I originally ignored systemd-networkd because I keep
> hearing that it's only for simple networks, but after looking at the man page
> it appears that it can do (almost) everything that I needed it to, although I'm
> not sure about dummy device support.  I need that for MATLAB, which stupidly
> requires the presence of an interface named eth0, whose MAC address it verifies
> during its licence check.  This became a problem after renaming my network
> devices following the news entry 2013-03-29-udev-upgrade ("Upgrading udev to
> version >=200").
> 
> What surprised me was that netifrc doesn't seem to integrate dummy devices
> properly, i.e., it doesn't seem to be possible to rename them in
> /etc/conf.d/net directly.  I implemented that via an appropriate call to ip in
> an /etc/local.d file, and had "net.eth0" depend on "local".  So /etc/conf.d/net
> looked like this:
> 
>     config_eth0="172.16.1.1/24"
>     mac_eth0="00:18:f3:97:17:72"
>     rc_net_eth0_provide="!net"
>     rc_net_eth0_need="local"
> 
> And the local.d script executed
> 
>     ip link set dev dummy0 name eth0
> 
> In comparison, the netctl configuration has everything in one place:
> 
>     # cat /etc/netctl/dummy
>     Description='A dummy interface (for MATLAB)'
>     Interface=eth0
>     Connection=dummy
>     IP=static
>     Address=172.16.1.1
>     IPCustom=( 'link set dev eth0 address 00:18:f3:97:17:72' )
> 
> It also takes care of loading the dummy kernel module, so I don't don't need an
> /etc/modules-load.d/ entry, whereas with netifrc/OpenRC I needed to load it
> manually via /etc/conf./modules (though maybe that has changed?).

A few days ago I finally got around to giving systemd-networkd a whirl, as I
said I would in the sub-thread started by Rich.  It turns out that it fulfils
the needs of my computers just fine, and has (together with systemd-resolved)
fully replaced netctl.  The only thing I'm not sure of is how extensive IPv6
support is.  The man page suggests that only DHCPv6 is supported, but not
stateless configuration.  Not that my LAN has IPv6, but it'd be nice to know
how future proof it is.

(This was made a tiny bit easier since I recently got rid of most of the server
functionality of my desktop, i.e., it has stopped acting as a router.  So I'm
only using the built-in ethernet port now, and will remove the other soon.)

I also switched the laptop over, which was using NetworkManager, but now uses
wpa_supplicant directly with systemd-networkd on top (basically swapping out
one upper layer for another).  It required a small bit of configuration so that
wpa_gui had a socket to connect to, and so that it could save the configuration
file, but that was just two additional lines (along with two minimal *.network
files, although now that I think about it I'm not even sure whether they're
needed). It appears to work as fine as it did when running NetworkManager, so
I'm happy to be rid of NM.

One thing I noticed about systemd-networkd is that it, too, follows the theme
of being designed around dynamic changes, namely network/device
addition/removal.  This becomes apparent when you learn that you can use wild
cards to match devices.  For example, when using the new udev network device
naming scheme (aka "predictable names"), "en*" matches all ethernet ports and
"wl*" matches all wireless ports.  So that's interesting, I think, and I'm
curious to see how systemd-networkd will evolve.

I would now like to take a moment to mourn the unmerging of dhcpcd, which
fulfilled its task of configuring the IP address (and other associated tasks) of
my computers marvellously for over 8 years (first emerged on: Wed Jan 10
14:44:09 2007).

Done?  Great, now let's celebrate the unmerging of NetworkManager :-D .
*phweeee* *balloon popping* *other generic party sounds*

> == Stuff I liked ==
> 
[...]
> == Questions ===
> 
> === ACPID: needed or not? ===
> 
> Does acpid provide anything that systemd does not, and if so, what kind of
> "conflicts" might I see?  The Gentoo Wiki page says that acpid is likely not
> needed.
> 
> FWIW, I already unmerged it and have not noticed any missing functionality,
> even after over a week of regular usage.  So I'm tending towards "no, not
> needed".

I have not noticed anything amiss in the many months I have been using my
systems without acpid.  So my "tendency" has morphed into "assuredness".

> === Timers ===
> 
> Can a systemd timer depend on a mount point such that it waits until the mount
> point exists before running?  Or will it fail after a timeout?  I want to
> research this myself, but haven't gotten around to it yet.
> 
> The problem I have is that my external HDD does not come up properly on cold
> boot, so that I have to unplug it and plug it back in in order for the kernel
> to fully initialise it and for it to mount, which is problematic for my backup
> fcron jobs, since they have &bootrun set. This means that the backup script
> will fail, unless I'm a) fast enough at re-plugging the HDD, and b) fast enough
> at logging in (so that my automounter mounts the HDD.  I will then have to
> manually re-run it (i.e., fcrondyn -x "run <ID>") or wait for the next time
> it's supposed to run.
> 
> Naturally, I would like a more robust system than that, and hope that systemd
> timers can make my life easier here.

I ended up asking specifically about the HDD problem on gentoo-user, and was
redirected to the linux-usb ML.  I already mentioned the final diagnosis on
gentoo-user, but haven't repeated it here yet:

    "After diagnosing the problem with the help of Alan Stern and Matthew
    Dharm, the conclusion was that the sector size translation must be
    triggered by some interaction between the drive and the BIOS.  The test I
    performed to determine this was:

    - unplug the drive,
    - turn on the computer,
    - plug the drive in once GRUB has started, and
    - boot the kernel,

    after which the drive worked fine.  So this does not seem to be anything
    that can be fixed in the kernel.  Perhaps if I had an EFI compatible
    mainboard, I wouldn't be seeing this problem, but I don't, so I can't even
    try it out :-/ ."

So, as sad as it is, my "solution" is to follow those four basic steps whenever
I turn my computer on, and then I don't get any problems.

[...]
> === Suspend on the desktop ===
> 
> Like I mentioned above, my Desktop can *almost* suspend reliably, after trying
> it out once every year or two (it's over 8 years old).  Mostly it would just
> not wake back up.  The latest status (before systemd) was that the kernel
> crashed after waking up (but I think that was a known bug that was fixed in the
> meantime).
> 
> Now with systemd, it wakes back up properly, but one of the soundcard drivers
> (ice1724) is apparently unreliable, so that sound stops working after wakeup.
> Plus, access to the soundcard was blocked by rtkit.  I believe what one would
> normally do is unload the module before suspending, but the only way I could
> find to tell systemd to do that requires creating a file in /usr (as done here,
> for example: http://forums.fedoraforum.org/showthread.php?t=294065).  Is there
> a better way?

Well, I ended up doing this, and suspend/resume *almost* works, but still not
quite: I couldn't access my USB3 HDD as a regular user anymore after resume, and
it seems that my desktop shuts itself off after a while.  Although maybe that
has to do with the empty CMOS battery? Or a failed hibernation attempt?  Just
some random ideas.

Again, I'm not pushing this, I'm just trying it out every now and then.  My
main motivation here is to waste as little electricity as is possible with this
old desktop.  What I would like to be able to do is to pause whatever I'm
doing, suspend, do something else for a while, and then resume, *without*
having to restart applications.  (Even better would be to buy a new, more energy
efficient computer, but I cannot afford that right now.)

@Duncan: it seems to me that the btrfs RAID has yet to be a problem during
resume (doesn't mean it won't be, but that's what I've seen so far).

[...]

Greetings
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Marc Joliet posted on Wed, 20 May 2015 10:01:13 +0200 as excerpted:

> A few days ago I finally got around to giving systemd-networkd a whirl,
> as I said I would in the sub-thread started by Rich.  It turns out that
> it fulfils the needs of my computers just fine, and has (together with
> systemd-resolved) fully replaced netctl.  The only thing I'm not sure of
> is how extensive IPv6 support is.  The man page suggests that only
> DHCPv6 is supported, but not stateless configuration.  Not that my LAN
> has IPv6, but it'd be nice to know how future proof it is.

I don't recall whether you mentioned whether you're running stable or 
~arch, and I didn't see mention of the version of systemd you're running 
now, but FWIW...

I'm ~arch, but am still on systemd-218 (-r3), while 219 is latest ~arch.  
This is for two reasons you may find interesting, one of which pertains 
to networkd and thus to the quoted bit, above:

Networkd: 219's networkd apparently breaks at least some (maybe static-IP-
only?) IPv4 only setups.  There's a complaint about lack of IPv6 support 
(duh! it's disabled on purpose as I don't use it, to avoid having to 
worry about securing it!), and the interface is not brought up.  The 
upstream bug (filed by someone else, with another report as well before 
mine, making me the third person reporting being affected) and gentoo bug 
(mine) are:

https://bugs.freedesktop.org/show_bug.cgi?id=90103

https://bugs.gentoo.org/show_bug.cgi?id=548380

So far no comment from upstream, but at least I know others are seeing it 
too.  The original reporter had disabled IPv6 with a kernel boot 
parameter, which hints at a more mainstream distro with a prebuilt binary 
kernel and modules shipped, while I have CONFIG_IPV6 unset in my of 
course custom-configured-and-built kernel here on gentoo.

So contrary to your wondering, IPv4 support in the present and near-
future may actually be a rather more pressing concern than IPv6 in the 
longer-term future. =:^(

Hopefully that bug will be fixed for 220, as broken network connectivity 
is rather critical, but the lack of even a developer comment in the month 
since filing isn't particularly encouraging.  But I have a static IP 
assigned and since I switched to systemd a version or two before the 
introduction of networkd, and was using a simple custom network.service 
unit at that time (not too difficult and easily enough googled, with a 
static IP), I figure worst-case, I simply dig that up again...  Tho I 
really can't imagine them leaving IPv4 broken, even if it's only for 
people who don't have IPv6 enabled.

One reporter did say he was able to bring up his network manually.  I 
didn't try that; I simply reverted to 218 and have masked succeeding 219+ 
versions after trying them and not finding a fix.  But that does indicate 
that my alternative network.service approach should work.


Meanwhile, the second bug affecting me in systemd-219...

Btrfs and tmpfiles.d: systemd-219 was supposed to add better btrfs 
support, since btrfs and in particular its subvolume support figures so 
strongly in the systemd "big picture" plan.  Unfortunately, systemd's 
first rollout of btrfs specific features has a few bugs, the one I ran 
across involving the new tmpfiles.d btrfs specific features.

In 219, systemd/tmpfiles.d introduces support for a new type key-letter, 
"v".  On "legacy" filesystems, "v" is supposed to act just like "d", 
create the given directory if it doesn't exist yet.  On btrfs, however, 
"v" creates a subvolume at that location instead, relying on the fact 
that btrfs subvolumes normally function much like directories, except 
with a few btrfs-specific features that normal directories don't have.

And 219 ships a couple tmpfiles.d dropin files that make use of "v" in 
place of "d", creating subvolumes on new btrfs where they (or 
directories) don't already exist, while it creates directories on 
"legacy" filesystems.

The problem?  An attempted directory-create of an existing directory, on 
a read-only filesystem (my / is kept read-only by default, I only mount 
it read-write to update it), succeeds.  An attempted subvolume create, 
where there's an existing (normal) directory, on a read-only (btrfs) 
filesystem, fails.

With that failure, systemd-tmpfiles-setup.service, run at boot (pulled in 
by sysinit.target), fails.

https://bugs.freedesktop.org/show_bug.cgi?id=90281

Status: resolved/fixed, with the git commit listed/linked, less than two 
weeks after filing.  =:^) Tho of course a release with the fix hasn't yet 
been made, and a fix in under two weeks there without even a comment in 
over a month on the IPv4-only bug doesn't bode so well for the latter.
=:^(

Of course a quick edit of the affected tmpfiles.d files, turning "v" to 
"d" where appropriate, would work around this issue, and now there's a 
patch-fix I could apply pretty easily as well, but I haven't bothered due 
to the above networkd bug.  I've simply masked each new 219+ ebuild after 
trying it to see if the two bugs are fixed yet, and thus remain on 218 
for now.

So while 218 worked well for me, 219 has been rough and remains masked, 
here.  We'll see how 220 turns out, when it's released and a gentoo ebuild 
for it becomes available...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Rich Freeman]

On Wed, May 20, 2015 at 6:44 AM, Duncan <1i5t5.duncan@cox.net> wrote:
>
> Status: resolved/fixed, with the git commit listed/linked, less than two
> weeks after filing.  =:^) Tho of course a release with the fix hasn't yet
> been made, and a fix in under two weeks there without even a comment in
> over a month on the IPv4-only bug doesn't bode so well for the latter.

I don't think systemd has any kind of stable release strategy, and I
think that is going to become a problem.  There are a lot of cases
where there are dramatic behavior changes or big feature changes, and
they're mixed in with fixes.

So, you can't use 219 because it lacks fixes to some bugs.
Systemd-220 will probably fix those, and change who-knows-what else.

They're relying on distros to backport fixes for them, essentially.
Gentoo has been doing just that, but this just means that Gentoo's
stable version of systemd is different from Debian or Arch's, and so
on.

For such a critical piece of software they really ought to start some
fixes branches with minor releases.  They don't necessarily need to
have 5-year LTS releases (though I'm sure they'd be appreciated
considering the nature of the software), but having some minor
releases for the last major version or two certainly wouldn't hurt.

-- 
Rich

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Rich Freeman posted on Wed, 20 May 2015 07:22:39 -0400 as excerpted:

> I don't think systemd has any kind of stable release strategy, and I
> think that is going to become a problem.  There are a lot of cases where
> there are dramatic behavior changes or big feature changes, and they're
> mixed in with fixes.

You are AFAIK correct, and I believe it's deliberate, as it is known to 
be with the binary journal format, for instance.

That bothered me too, so much so that I had originally thought that while 
I'd eventually switch to systemd, I'd sit out until the featuritis slowed 
down and the product stabilized to some degree... tho as some would argue 
happened to pulse-audio (I'm on the sidelines on that one as I've never 
had it installed, here, as I've simply never found I needed it strongly 
enough to do the research necessary to be comfortable installing and 
properly configuring it), that may not happen until Lenart loses interest 
and finds some other project to disrupt the Linux ecosystem with.

But at some point I realized that I was on live-git openrc-9999 in 
ordered to properly follow individual git commits (I had problems earlier 
when I had been on standard ~arch, because the changelog wasn't detailed 
enough to properly trace any problems I had and it was thus much more 
work than it should have been, and than it was once I switched to live-
git openrc), and regardless of how unstable systemd's releases turned out 
to be, at least they were releases... and if I had to do so for similar 
reasons, I could go live-git systemd-9999 as well.

Between that (#1), and once I started experimenting (when I had both 
installed for a time and could switch between them from grub2 with init= 
on the kernel commandline), (2) being /very/ impressed with the level of 
documentation systemd has in general, (3) being /very/ impressed with 
what it did to boot times, even compared to openrc's parallel-init 
option, and (4) finding once I actually got into it that I actually liked 
systemd's declarative style config default, in part because despite a 
declarative config style default, there was still the flexibility to call 
scripts from a unit if it was actually found to be necessary, I decided 
to keep systemd, and ultimately, to unmerge openrc, as I was simply no 
longer keeping up with its config, such that even if I /did/ find I 
needed it, I'd be afraid to run it for fear of what running untested 
updates would now do to the system.  (And anyway, I can always boot to 
the emergency or rescue targets directly, just as one would previously 
boot to initlevel 1/single, and failing that, I could still boot
init=/bin/bash, which in fact I have a grub2 menu entry for.)

So while systemd's failure to have a decent stable/unstable releases 
policy, as well as the continued featuritis, continues to bother me, 
because gentoo /does/ keep older versions around for awhile (and because 
being gentoo, if old versions are removed from the tree, I can always 
dredge the old version from the installed package database or from my 
binpkgs, and put it in my overlay), it's not the problem it might 
otherwise be.  In fact, this whole incident actually supports that... 
because it's gentoo, I actually /have/ 218 (and older versions) still 
available to me, despite the fact that 219 is current-latest ~arch.

So in reality systemd hasn't been any worse than openrc was for me, and 
in a number of ways (including documented config, real speed, cross-
distro standardization so google's more effective, /and/ not signficantly 
more and possibly less show-stopping bugs than openrc), it has actually 
been better, /despite/ the lack of a coherent stable/unstable release 
plan.

Which doesn't mean it wouldn't be better still, if they'd adopt such a 
coherent stable/unstable release plan... nor does it mean I won't at some 
point decide I need to be on live-git to get better commit-level 
granularity on bugs when I see 'em, the better to track them down and 
exterminate them, just as I was for openrc.

There are, however, two other concerns I have about systemd, one in a 
purely practical openrc context, one in a much more general "system 
knowledge bootstrapping ability" context:

1) (openrc context):  Given my experience running live-git openrc, and 
the fact that judging from the bugs filed against it, at least during 
part of the time I ran it, I was the only one (beyond the openrc devs 
themselves) doing so, or at least the only one doing so and filing 
bugs... my having switched to systemd certainly could not have had a good 
effect on openrc pre-release live-git testing and thus bug squashing 
before they ever hit a release at all.

Oh, well...  it's an "SEP", "somebody else's problem", now...  (And to be 
fair, it /did/ appear that toward the end, there was an uptick in openrc 
live-git bugs filed by others too, so others were evidently running it by 
the time I quit.  But still, it was only a handful, say five or so, in 
which case a loss of just one is a loss of 20% of your pre-release 
testing coverage!)

2) (larger philosophical/practical context):  Back in 2001/2002 when I 
got serious about and switched to Linux, I read a couple books, but I 
actually got my practical hands-on shell experience by rewriting several 
of the Mandrake init-scripts, including the core sysinitrc (or whatever 
it was called, that was nearly a decade and a half again, after all!).

I can't believe I was the only one.

As a result, one of the nagging fears I have about systemd, despite all 
the improvements I believe it does bring, is that this early gateway to 
practical shell knowledge and experience is literally disappearing before 
our eyes, and people trying to become Linux CLI/shell literate today are 
going to have a much harder time than people of my Linux generation did, 
because there's far less shell scripting actually available for them to 
work on, and it's far less prominently placed, making it much harder to 
simply stumble upon, as I basically did.

Between that, and the transparency of a shell-based system init that 
they're losing as well, today's newbies may well find it far harder to 
get in as deeply, as quickly, as I did, and the wonders of system bootup 
may as a result remain as practically opaque to them as an MS Windows 
boot.

That would be, and is, a sad thing.  But of course I know the Linux boot 
process pretty well now, the more so since I now have the experience of a 
sysv/shell-script-based init from multiple distros, AND the far different 
systemd-based init, too.  And sad philosophical/practical concern or not, 
being in the knowledge clase now, it's not going to keep me away from the 
faster and more efficiently configured systemd boot. =:^\

(Meanwhile, FWIW, back in the day I knew my way around an MS Windows boot 
too, and in the day hacked solutions to a couple MS Windows driver 
issues, etc.  But never in the decade I was on MS, both DOS and Windows, 
did I know its boot process to the degree that I knew that of Mandrake 
Linux, just a year after switching, and that too was just scratching the 
surface, compared to what I knew six months after installing Gentoo from 
stage-1, tho of course in the decade since I've learned far more still.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 4021 bytes --]

Am Thu, 21 May 2015 09:36:28 +0000 (UTC)
schrieb Duncan <1i5t5.duncan@cox.net>:

> Rich Freeman posted on Wed, 20 May 2015 07:22:39 -0400 as excerpted:
[...]
> So while systemd's failure to have a decent stable/unstable releases 
> policy, as well as the continued featuritis, continues to bother me, 
> because gentoo /does/ keep older versions around for awhile (and because 
> being gentoo, if old versions are removed from the tree, I can always 
> dredge the old version from the installed package database or from my 
> binpkgs, and put it in my overlay), it's not the problem it might 
> otherwise be.  In fact, this whole incident actually supports that... 
> because it's gentoo, I actually /have/ 218 (and older versions) still 
> available to me, despite the fact that 219 is current-latest ~arch.
> 
> So in reality systemd hasn't been any worse than openrc was for me, and 
> in a number of ways (including documented config, real speed, cross-
> distro standardization so google's more effective, /and/ not signficantly 
> more and possibly less show-stopping bugs than openrc), it has actually 
> been better, /despite/ the lack of a coherent stable/unstable release 
> plan.

I agree, though I've only used two systemd releases till now (216 and 218).

[...] 
> 2) (larger philosophical/practical context):  Back in 2001/2002 when I 
> got serious about and switched to Linux, I read a couple books, but I 
> actually got my practical hands-on shell experience by rewriting several 
> of the Mandrake init-scripts, including the core sysinitrc (or whatever 
> it was called, that was nearly a decade and a half again, after all!).
> 
> I can't believe I was the only one.
> 
> As a result, one of the nagging fears I have about systemd, despite all 
> the improvements I believe it does bring, is that this early gateway to 
> practical shell knowledge and experience is literally disappearing before 
> our eyes, and people trying to become Linux CLI/shell literate today are 
> going to have a much harder time than people of my Linux generation did, 
> because there's far less shell scripting actually available for them to 
> work on, and it's far less prominently placed, making it much harder to 
> simply stumble upon, as I basically did.

I don't know... I learned basically all of my shell scripting from trying to
solve problems I was seeing and simply because I wanted to try out something.
I didn't touch init scripts until I had to (for work), and by then OpenRC
supported its own declarative style (see openrc-run(8)), so the amount of
shell *scripting* involved was minimal (and then we switched that computer to
Fedora, and hence to systemd).

Anyway, it seems to me that "scratching your own itch" was the more basic
motivation you had for learning shell scripting via init scripts.

> Between that, and the transparency of a shell-based system init that 
> they're losing as well, today's newbies may well find it far harder to 
> get in as deeply, as quickly, as I did, and the wonders of system bootup 
> may as a result remain as practically opaque to them as an MS Windows 
> boot.

I would argue that in this case people at least have the source code.  But more
importantly, the *principal* of what happens is still not very different:
commands get executed with particular arguments and a particular environment.
For me, personally, the latter is the least transparent, I suppose, but then
again, "systemd show" shows you all the properties of a unit, which (in
combination with the systemd documentation) gives you pretty much all of the
information you need to learn what *exactly* is going on.

(Of course, this is just me trying to show an alternative view point, not
trying to make it look like this isn't a potential issue, even though from my
perspective it isn't.)

[...]

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

Am Thu, 21 May 2015 09:36:28 +0000 (UTC)
schrieb Duncan <1i5t5.duncan@cox.net>:

[...]
> I decided 
> to keep systemd, and ultimately, to unmerge openrc
[...]

I forgot to ask about this: how did you go about doing that?  I'm still waiting
for the following two bugs to be resolved:

https://bugs.gentoo.org/show_bug.cgi?id=504116
https://bugs.gentoo.org/show_bug.cgi?id=511500

Do you have none of the affected packages installed, or...?

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 1677 bytes --]

Am Sat, 23 May 2015 10:49:07 +0200
schrieb Marc Joliet <marcec@gmx.de>:

> Am Thu, 21 May 2015 09:36:28 +0000 (UTC)
> schrieb Duncan <1i5t5.duncan@cox.net>:
> 
> [...]
> > I decided 
> > to keep systemd, and ultimately, to unmerge openrc
> [...]
> 
> I forgot to ask about this: how did you go about doing that?  I'm still waiting
> for the following two bugs to be resolved:
> 
> https://bugs.gentoo.org/show_bug.cgi?id=504116
> https://bugs.gentoo.org/show_bug.cgi?id=511500
> 
> Do you have none of the affected packages installed, or...?

Partially answering this myself: you must be using gcc-config-1.8-r1, and have
none of the other still affected packages installed (e.g., java, hence no
java-config-wrapper).

Following bug #549508, I could put java-config-wrapper in package.provided and
unmask gcc-config-1.8-r1.  Then I should be able to put openrc in
package.provided and finally depclean it.  This should work as a temporary
solution until bugs #504116 and #511500 are resolved.

Does that sound sane?

The only two scary looking bugs I found on gcc-config are 547586 and 547962.
Bug #547586 is apparently completely benign (whitespace differences in "make
check" output), whereas bug #547962 caused gcc-config-1.8-r1 to be hard masked.
However, the underlying bug in gentoo-functions has apparently been fixed [0]
(according to its ChangeLog this was done in version 0.8, which is arch), so it
seems to me that the hard mask can be lifted.

[0] https://bugs.gentoo.org/show_bug.cgi?id=547962#c5

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Marc Joliet posted on Sat, 23 May 2015 10:49:07 +0200 as excerpted:

> Am Thu, 21 May 2015 09:36:28 +0000 (UTC)
> schrieb Duncan <1i5t5.duncan@cox.net>:
> 
> [...]
>> I decided to keep systemd, and ultimately, to unmerge openrc
> [...]
> 
> I forgot to ask about this: how did you go about doing that?  I'm still
> waiting for the following two bugs to be resolved:
> 
> https://bugs.gentoo.org/show_bug.cgi?id=504116
> https://bugs.gentoo.org/show_bug.cgi?id=511500
> 
> Do you have none of the affected packages installed, or...?

Good question!

Answer: "or..."  Very definitely "or...", as you will see below. =:^)

functions.sh is easy enough to fix, if you're willing to create a single 
symlink, manually.

$ equery belongs functions.sh
sys-apps/gentoo-functions-0.10 (/lib/gentoo/functions.sh)

The package gentoo-functions is /supposed/ to replace a couple gentoo-
specific files formerly found in openrc, thereby making openrc optional 
for those using other init systems (like systemd).  One of those files is 
functions.sh[1].

But the old /etc/init.d/functions.sh location was hardcoded into a lot of 
gentoo utilities over the years, some of which don't get updated very 
often especially for stable, and the new /lib/gentoo/functions.sh 
location, while more appropriate, still can't be found by some of these 
apps.  That's what the first bug is about.

Simple enough fix -- ensure the gentoo-functions package is installed (it 
should be as it's a dependency of enough stuff, including current glibc 
packages!), and do a manual symlink from the old location to the new 
one.  Viola! No openrc needed for THAT any more, and because the new 
location's file is still packaged, any updates to it should occur 
normally and will be simply picked up by anything still using the old 
location, since it's a symlink! =:^)


The other bug deals with the base-profile placing openrc in @system, 
since until the functions.sh bug is finally resolved, it really is a 
system dependency of a sort as it provides the old-location functions.sh, 
even if you're using something else as init system.

First let me caution that the below specific action is rather "out there" 
and cannot be expected to be supported by gentoo, tho that said, it 
hasn't caused /me/ any issues, running it for I'd guess a couple years 
now... and for sufficiently experienced gentooers it's not really as 
unreasonable as it might sound...

Warning out of the way...

What I've done here, actually quite some time before I switched to 
systemd, is negate my entire @system, so it's entirely empty.  I actually 
get a pretty dire looking warning about the empty @system when I run 
emerge --depclean, but when I negated @system, I did so one package at a 
time, and added packages to my @world list as necessary, where I really 
did need (or want) them and nothing else depended on them.

Of course this only works for gentooers with enough experience to tell 
what @system packages that would be removed when it's empty, are really 
needed after all.  I obviously had the required experience as I've been 
running @system-less for quite some time now, but it's obviously not for 
everyone.

Obviously for things like virtual/editor, I already had my editor of 
choice in @world, and thus could simply negate the virtual entirely.  In 
other cases, something else in @world has a dep on the @system package, 
so I didn't need to put that package in @world when I took it out of 
@system, at all.

And things like busybox... aren't really necessary at all.  In fact, I've 
actually never had it installed in the over a decade I've been on gentoo, 
as for whatever reason it wouldn't build when I originally installed, so 
I temporarily bypassed it as I knew it was an emergency thing, and that 
temporary thing just got longer and longer, until I decided I didn't need 
it at all.

Instead of something like busybox, I keep (at least) two root partitions 
available, my working copy and my backup.  The two partitions are the 
same size, and contain pretty much everything installed by portage (with 
a small exception for some state files in /var, since I keep root mounted 
read-only by default and they need to be writable).  Periodically, when 
the system seems to be stable, I'll blow away the backup root with a 
fresh mkfs, and then copy everything back onto it, creating a new backup 
that's a copy of the same working system setup that I'm using at that 
moment.  Of course a good sysadmin never considers a backup done until 
it's tested, so then I test-boot to it to ensure it works.

Meanwhile, in grub2, I have a menu entry that sets a variable used in the 
root= entry on the kernel commandline.  By default, that var points at 
the working root, but if I invoke that menu entry, it changes the var to 
point at the backup root instead, thus letting me boot either working or 
backup root, by just selecting an entry in the grub2 menu before I load 
the kernel, if I want to boot the backup.

So I don't need something like busybox, because in the event I screw up 
the working root and can't boot it, I can boot the backup, which is a 
copy of the very same working config I was using when I took it.  So 
while it might be a bit old if I haven't updated the backup in awhile, 
it's tested to boot properly, when I made the backup.

And actually, my working system and primary backup are on dual SSDs in 
btrfs raid1 mode, with a partition each for the working and primary 
backup, and I have another backup on my old spinning rust drive as well, 
on reiserfs in case btrfs bugs out on me and the working and primary 
backup are thus unusable, with yet another grub menu entry to set it up 
for boot.  Meanwhile, I have a separate grub installation and separate 
/boot, on all three drives, so as long as all three drives don't go out 
at once...

And of course ssh, still part of @system altho people running local 
systems really don't need it as part of @system, works just fine in 
@world, if you actually do want/need it.

Just some examples...

OK, but how does this @system negation actually /work/?

Simple enough.  The portage (5) manpage has the documentation, but 
briefly, profiles dirs can have packages files.  In these files are 
entries like this:

*sys-apps/busybox
*>=sys-apps/baselayout-2

The * indicates that this package should be added to the @system set.  
(In new enough portage and profiles there's also @profile set entries, 
see the manpage, but they haven't bothered me yet, here.)

And critically, in cascading profiles, which pretty much all profiles are 
these days, -* indicates a *REMOVAL* from the @system set.

Two additional missing pieces:

/etc/portage/profile/packages... is where you put your installation-
specific things, and where I put my negations.

Note the versioned baselayout-2 entry above.  These versioned entries 
must be negated with the identical version, or they don't end up 
negated.  So that baselayout entry would be negated like this:

-*>=sys-apps/baselayout-2


As I said, with a clean emerge --depclean --pretend, I negated one or a 
small handful at a time, then did an emerge --depclean --pretend and 
noted what the new @system removals had now cleared for removal as 
nothing else depended on them.  Some, such as virtual/editor, I already 
had an appropriate editor in @world for and thus removal of the virtual 
was fine.  Others, such as sys-apps/busybox, I never had installed in the 
first place.  Others, including sys-apps/man-pages, I added to @world, 
and still others, including sys-fs/e2fsprogs, were deps of something else 
and thus didn't need a specific @world entry to keep depclean from 
removing them.

And of course, the two that triggered the subthread, openrc and 
baselayout.  Openrc I originally kept in @world as I was using it for init 
when I did the @system negation.  Since it was then in @world, not in 
@system, when I switched to systemd for init, it was easy enough to 
unmerge it, just as I would anything else in @world.

Meanwhile, systemd itself depends on baselayout, and I'd guess openrc 
does as well altho I don't remember for sure and am lazy to look ATM, so 
it stays on the system even tho it's not in @system any longer.


Finally, one additional thing to note about @system, that's different 
when you don't have it.  If you try to unmerge a package in @system, you 
get a dire warning.  Obviously, if your @system is empty and the same 
package is simply listed in @world or a dep of something in @world, you 
won't get that same kind of dire warning trying to unmerge it.  For 
experienced gentooers that should be no big deal, as they wouldn't think 
of trying to unmerge really critical stuff like glibc anyway.  But for 
less experienced gentooers, those dire warnings might save their *** once 
in awhile, making them useful to have.  If you're at all unsure whether 
you need such "safety railings", emptying @system entirely probably isn't 
something you should be considering.


But, just because you aren't emptying @system entirely, doesn't mean you 
can't negate specific @system package entries, if you're sure you don't 
need them.  For systemd users who don't want/need the fallback to openrc 
in case systemd fails to boot, or where the openrc config is getting 
dangerously outdated due to lack of use such that even if systemd breaks, 
they'd be afraid to simply set init to run openrc instead, lest it break 
their system further with an unconfigured and untested boot, and who have 
or will take care of that /etc/init.d/functions.sh file via manual 
symlink, openrc may well be one such entry to consider negating.


....

So you see why I said "or..." DEFINITELY "or...", above! =:^)

---
[1] gentoo-functions file replacements:  In addition to functions.sh, the 
other is consoletype, executable and manpage, see the results of equery 
files gentoo-functions.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 2518 bytes --]

Am Sat, 23 May 2015 10:41:03 +0000 (UTC)
schrieb Duncan <1i5t5.duncan@cox.net>:

> Marc Joliet posted on Sat, 23 May 2015 10:49:07 +0200 as excerpted:
> 
> > Am Thu, 21 May 2015 09:36:28 +0000 (UTC)
> > schrieb Duncan <1i5t5.duncan@cox.net>:
> > 
> > [...]
> >> I decided to keep systemd, and ultimately, to unmerge openrc
> > [...]
> > 
> > I forgot to ask about this: how did you go about doing that?  I'm still
> > waiting for the following two bugs to be resolved:
> > 
> > https://bugs.gentoo.org/show_bug.cgi?id=504116
> > https://bugs.gentoo.org/show_bug.cgi?id=511500
> > 
> > Do you have none of the affected packages installed, or...?
> 
> Good question!
> 
> Answer: "or..."  Very definitely "or...", as you will see below. =:^)
> 
> functions.sh is easy enough to fix, if you're willing to create a single 
> symlink, manually.
> 
> $ equery belongs functions.sh
> sys-apps/gentoo-functions-0.10 (/lib/gentoo/functions.sh)
> 
> The package gentoo-functions is /supposed/ to replace a couple gentoo-
> specific files formerly found in openrc, thereby making openrc optional 
> for those using other init systems (like systemd).  One of those files is 
> functions.sh[1].
> 
> But the old /etc/init.d/functions.sh location was hardcoded into a lot of 
> gentoo utilities over the years, some of which don't get updated very 
> often especially for stable, and the new /lib/gentoo/functions.sh 
> location, while more appropriate, still can't be found by some of these 
> apps.  That's what the first bug is about.
> 
> Simple enough fix -- ensure the gentoo-functions package is installed (it 
> should be as it's a dependency of enough stuff, including current glibc 
> packages!), and do a manual symlink from the old location to the new 
> one.  Viola! No openrc needed for THAT any more, and because the new 
> location's file is still packaged, any updates to it should occur 
> normally and will be simply picked up by anything still using the old 
> location, since it's a symlink! =:^)
[...]

That is certainly an interesting approach, since it avoids any upgrades.  Then
I would only have to deal with openrc.

[snip empty @system approach]

Wow, that's certainly an interesting approach!  It's definitely not for me,
though.  While I appreciate the control Gentoo gives me, I don't need *that*
kind of control.

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Rich Freeman]

On Sat, May 23, 2015 at 7:11 AM, Marc Joliet <marcec@gmx.de> wrote:
> [snip empty @system approach]
>
> Wow, that's certainly an interesting approach!  It's definitely not for me,
> though.  While I appreciate the control Gentoo gives me, I don't need *that*
> kind of control.
>

I'd actually like to see Gentoo evolve until we reach a point where
this sort of thing is easy to do safely.  Right now @system is a
mish-mash of a bunch of stuff and it is hard to pick out stuff like
openssh which is trivially removable from stuff like openrc which has
surprising reverse deps from stuff like glibc which obviously isn't
going anywhere anytime soon.

I'd really like to see a few virtuals created (perhaps nesting) that
chop this up into categories like recommend-default packages, stuff
required for POSIX, and so on.  We can still give users a good default
experience, and we can still avoid having ebuilds having to list out a
laundry-list of deps, but we can also do things like allow more
parallel-building and make it easier to remove stuff.

-- 
Rich

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Rich Freeman posted on Sat, 23 May 2015 07:37:26 -0400 as excerpted:

> I'd actually like to see Gentoo evolve until we reach a point where this
> sort of thing is easy to do safely.  Right now @system is a mish-mash of
> a bunch of stuff and it is hard to pick out stuff like openssh which is
> trivially removable from stuff like openrc which has surprising reverse
> deps from stuff like glibc which obviously isn't going anywhere anytime
> soon.
> 
> I'd really like to see a few virtuals created (perhaps nesting) that
> chop this up into categories like recommend-default packages, stuff
> required for POSIX, and so on.  We can still give users a good default
> experience, and we can still avoid having ebuilds having to list out a
> laundry-list of deps, but we can also do things like allow more
> parallel-building and make it easier to remove stuff.

What about the new @profile vs @system stuff?  I don't believe I've seen 
anything move to @profile yet, tho I'm not /exactly/ sure what sort of 
indication I'd get, but in theory at least, @profile was supposed to at 
least avoid the parallel-build blockages that @system does for safety.

Anything practical come of the @profile/@system split yet?  It didn't 
turn out to be a bad solution to the problem did it?  Other opinions and/
or experience on/with it?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 3089 bytes --]

Am Sat, 23 May 2015 13:11:56 +0200
schrieb Marc Joliet <marcec@gmx.de>:

> Am Sat, 23 May 2015 10:41:03 +0000 (UTC)
> schrieb Duncan <1i5t5.duncan@cox.net>:
> 
> > Marc Joliet posted on Sat, 23 May 2015 10:49:07 +0200 as excerpted:
> > 
> > > Am Thu, 21 May 2015 09:36:28 +0000 (UTC)
> > > schrieb Duncan <1i5t5.duncan@cox.net>:
> > > 
> > > [...]
> > >> I decided to keep systemd, and ultimately, to unmerge openrc
> > > [...]
> > > 
> > > I forgot to ask about this: how did you go about doing that?  I'm still
> > > waiting for the following two bugs to be resolved:
> > > 
> > > https://bugs.gentoo.org/show_bug.cgi?id=504116
> > > https://bugs.gentoo.org/show_bug.cgi?id=511500
> > > 
> > > Do you have none of the affected packages installed, or...?
> > 
> > Good question!
> > 
> > Answer: "or..."  Very definitely "or...", as you will see below. =:^)
> > 
> > functions.sh is easy enough to fix, if you're willing to create a single 
> > symlink, manually.
> > 
> > $ equery belongs functions.sh
> > sys-apps/gentoo-functions-0.10 (/lib/gentoo/functions.sh)
> > 
> > The package gentoo-functions is /supposed/ to replace a couple gentoo-
> > specific files formerly found in openrc, thereby making openrc optional 
> > for those using other init systems (like systemd).  One of those files is 
> > functions.sh[1].
> > 
> > But the old /etc/init.d/functions.sh location was hardcoded into a lot of 
> > gentoo utilities over the years, some of which don't get updated very 
> > often especially for stable, and the new /lib/gentoo/functions.sh 
> > location, while more appropriate, still can't be found by some of these 
> > apps.  That's what the first bug is about.
> > 
> > Simple enough fix -- ensure the gentoo-functions package is installed (it 
> > should be as it's a dependency of enough stuff, including current glibc 
> > packages!), and do a manual symlink from the old location to the new 
> > one.  Viola! No openrc needed for THAT any more, and because the new 
> > location's file is still packaged, any updates to it should occur 
> > normally and will be simply picked up by anything still using the old 
> > location, since it's a symlink! =:^)
> [...]
> 
> That is certainly an interesting approach, since it avoids any upgrades.  Then
> I would only have to deal with openrc.
> 
> [snip empty @system approach]
> 
> Wow, that's certainly an interesting approach!  It's definitely not for me,
> though.  While I appreciate the control Gentoo gives me, I don't need *that*
> kind of control.

Well, I did end up doing this:

% cat /etc/portage/profile/packages
-*sys-apps/openrc

and creating the symlink as suggested.

I also added java-config-wrapper to package.provided as I originally suggested,
following the declaration of obsolescence in bug #549508.

So far everything seems to be working fine (i.e., gcc-config and
java-config-2 both work).

Thanks
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Rich Freeman posted on Wed, 20 May 2015 07:22:39 -0400 as excerpted:

> So, you can't use 219 because it lacks fixes to some bugs. Systemd-220
> will probably fix those, and change who-knows-what else.

FWIW, systemd-220 is out upstream now, but there's not an in-tree gentoo 
ebuild for it yet (well, as of a bit under a day ago when I last updated).

Here's a link to the article where I found out about it (it was on the 
lxer feed I subscribe to via claws-mail's feed-reader addon, but that was 
just a redirect to the below article, which I'm linking directly instead):

http://www.itrunsonlinux.com/desktopos/systemd-v220-released/

The article pastes in the 220 changelog, apparently verbatim (minus the 
"contributors to this version" paragraph systemd normally puts at the end 
of each version's changelog).

That should have the patch for my btrfs tmpfiles.d "v" vs. "d" bug, as it 
was committed before the 220 release, tho I've obviously not tested it 
yet.

And while there's still no developer comment on the upstream IPv4 bug I'm 
CCed on, and as stated, no gentoo 220 yet, there's a patch from upstream 
for me to try on the gentoo IPv4 only bug I filed.  Presumably, if it's 
from upstream, that should be in 220 as well, and if it works...

Off for a 1 AM sync and some testing, now... if I don't get too sleepy to 
think straight first.  Still 219 with those patches applied if 220 isn't 
in-tree yet when I sync.  I'll post back when I have some results.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-amd64] Re: Systemd migration: opinion and questions

[Duncan]

Duncan posted on Sat, 23 May 2015 08:17:59 +0000 as excerpted:

> Off for a 1 AM sync and some testing, now... if I don't get too sleepy
> to think straight first.  Still 219 with those patches applied if 220
> isn't in-tree yet when I sync.  I'll post back when I have some results.

Well, back with a mixed report... still on 219_p112 as 220 still hadn't 
made it to the tree.

Upstream's patch for the tmpfiles/subvolumes/dirs thing worked perfect. 
=:^)

The gentoo-bug proposed patch to get IPv4-only working again... not so 
much.  It addressed one complaint in the log, but that wasn't the major 
problem, apparently, and networkd still doesn't bring up the network 
properly.

But, while I was testing, I tested something that another poster had 
noted on the upstream IPv4 bug -- networkd appears to be configuring the 
interface, just not bringing it up.  So a simple ifconfig <iface> up 
(what I used, or equivalent IP command, which is what he used) brings the 
interface up, configured as it should be. =:^)


Between the two, I think I'll stick with 219_p112 instead of reverting 
back to 218-r3, for now.  The tmpfiles.d thing is fixed by the upstream 
patch I dropped into /etc/portage/patches/sys-apps/systemd before the 
rebuild, and I can do a manual ifconfig up after reboot for now, or setup 
a quick service to automate it if I get tired of doing the manual ifconfig 
thing.

So mixed report, but there's progress, and a workaround for the remaining 
issue, so 219_p112 with additional patches and a manual workaround it is, 
for now.  =:^/

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Systemd migration: opinion and questions

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 1641 bytes --]

Am Wed, 20 May 2015 10:44:58 +0000 (UTC)
schrieb Duncan <1i5t5.duncan@cox.net>:

> Marc Joliet posted on Wed, 20 May 2015 10:01:13 +0200 as excerpted:
> 
> > A few days ago I finally got around to giving systemd-networkd a whirl,
> > as I said I would in the sub-thread started by Rich.  It turns out that
> > it fulfils the needs of my computers just fine, and has (together with
> > systemd-resolved) fully replaced netctl.  The only thing I'm not sure of
> > is how extensive IPv6 support is.  The man page suggests that only
> > DHCPv6 is supported, but not stateless configuration.  Not that my LAN
> > has IPv6, but it'd be nice to know how future proof it is.
> 
> I don't recall whether you mentioned whether you're running stable or 
> ~arch, and I didn't see mention of the version of systemd you're running 
> now, but FWIW...

I'm arch, so running systemd-218.

> I'm ~arch, but am still on systemd-218 (-r3), while 219 is latest ~arch.  
> This is for two reasons you may find interesting, one of which pertains 
> to networkd and thus to the quoted bit, above:
> 
[Snip two bug descriptions]

Damn, that sounds bad.  However, I'm running stable, so won't be affected.

I do agree with both you and Rich, though, that systemd really ought to have a
stable branch.  Their release workflow appears to me to be much like that of
the linux kernel, only without the stable trees.  Honestly, I would be
surprised if they didn't have the developer resources to provide this.

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 819 bytes --]