[gentoo-user] OT: cleanup after USB backup drive unplugged?

[gentoo-user] OT: cleanup after USB backup drive unplugged?

[Grant Edwards]

[This has nothing to do specifically with Gentoo.]

What cleanup actions would you have put in a script to be triggered by
udev when a USB or Firewire backup drive has been unplugged?

The external Firewire drive I used for nightly backups died yesterday.

I replaced it with a USB3 drive, so I needed to update the udev rules
that automatically mount it and then "umount" it when it's removed.
Both rules are firing when they should:

KERNEL=="sd?1",\
 SUBSYSTEMS=="scsi",\
 ENV{ID_SERIAL}=="WD_My_Passport_259F_57584A314143353435594B54-0:0",\
 ACTION=="add",\
 SYMLINK+="passport",\
 RUN+="/bin/mount -text4 -o atime /dev/%k /extbackup"

KERNEL=="sd?1",\
 ACTION=="remove",\
 ENV{ID_SERIAL}=="WD_My_Passport_259F_57584A314143353435594B54-0:0",\
 RUN+="/usr/local/bin/myumount /extbackup"

Here's the embarassing part: The /usr/local/bin/myumount script went
missing (backup drive is dead), and I can't recall exactly what it
did.  Obviously, one should never unplug the drive while it's mounted,
but if that _does_ happen, what would one put in myumount to mitigate
the situation.

The only think I can think of is to do a "umount -l".

-- 
Grant Edwards               grant.b.edwards        Yow! How many retured
                                  at               bricklayers from FLORIDA
                              gmail.com            are out purchasing PENCIL
                                                   SHARPENERS right NOW??

Re: [gentoo-user] OT: cleanup after USB backup drive unplugged?

[Adam Carter]

[-- Attachment #1: Type: text/plain, Size: 390 bytes --]

>
> I replaced it with a USB3 drive, so I needed to update the udev rules
> that automatically mount it and then "umount" it when it's removed.
>

Pretty sure you'd risk filesystem corruption by not umounting before you
remove the device. Did it used for force an fsck on each mount because the
filesystem was "dirty"? If not, i have a fundamental misunderstanding of
how this stuff works.

[-- Attachment #2: Type: text/html, Size: 646 bytes --]

Re: [gentoo-user] OT: cleanup after USB backup drive unplugged?

[Mick]

[-- Attachment #1: Type: text/plain, Size: 834 bytes --]

On Friday, 12 January 2018 09:58:17 GMT Adam Carter wrote:
> > I replaced it with a USB3 drive, so I needed to update the udev rules
> > that automatically mount it and then "umount" it when it's removed.
> 
> Pretty sure you'd risk filesystem corruption by not umounting before you
> remove the device. Did it used for force an fsck on each mount because the
> filesystem was "dirty"? If not, i have a fundamental misunderstanding of
> how this stuff works.

Yes, the sequence ought to be:

sync && umount /dev/<device> || eject /dev/<device>

I don't recall if eject includes the previous steps and therefore it is 
superfluous.

I find some applications are reluctant to let go of filesystems on pluggable 
devices and umount can fail.  In this case --lazy option of umount should 
work:

umount -l /dev/<device>

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] OT: cleanup after USB backup drive unplugged?

[Rich Freeman]

On Fri, Jan 12, 2018 at 6:39 AM, Mick <michaelkintzios@gmail.com> wrote:
> On Friday, 12 January 2018 09:58:17 GMT Adam Carter wrote:
>>
>> Pretty sure you'd risk filesystem corruption by not umounting before you
>> remove the device. Did it used for force an fsck on each mount because the
>> filesystem was "dirty"? If not, i have a fundamental misunderstanding of
>> how this stuff works.
>
> Yes, the sequence ought to be:
>
> sync && umount /dev/<device> || eject /dev/<device>
>
> I don't recall if eject includes the previous steps and therefore it is
> superfluous.
>

Actually, all but umount are superfluous for a USB device.

umount already ensures that all writes are written to disk - the
system call does not return until the disk is completely settled.  All
a sync would do is force all your other filesystems to also flush
their write caches, which could cause performance issues and
potentially make it take longer to flush the drive you actually want
to remove.  There is no need to run eject on a USB drive - just pull
the thing and udev will clean up the device nodes.  For other busses
like SATA there might be benefits to deleting the device before hot
swapping them, but I'm not sure if eject actually works on those.

If you umount the device, and wait for the command to terminate, then
you're fine removing it.

This is all somewhat orthogonal to your original question though -
these are all things that you ought to do before you remove the
device, not after.

One thing I tend to do with scripts for backups to external devices
that I don't intend to leave mounted is build the mount and umount
into the backup script itself.  I usually also include a check to
ensure that some file/directory exists which I expect to be on the
drive, which prevents the backup script from dumping a full backup
into your mountpoint if it isn't mounted (possibly on a filesystem
with insufficient space - actually, making /mnt a tmpfs with a tiny
RAM allotment might make sense for just this reason).  If you're
running your backups in a service this could also be stuff that goes
into the service config (esp with systemd you can make a mount a
pre-req and have it automatically unmounted on termination whether
successful or not; you could probably script the same sort of thing
with openrc, but there is less benefit there since you don't have
things like timer units to trigger them and they're not
containerized/etc).

-- 
Rich

[gentoo-user] Re: OT: cleanup after USB backup drive unplugged?

[Grant Edwards]

On 2018-01-12, Rich Freeman <rich0@gentoo.org> wrote:
> On Fri, Jan 12, 2018 at 6:39 AM, Mick <michaelkintzios@gmail.com> wrote:
>> On Friday, 12 January 2018 09:58:17 GMT Adam Carter wrote:
>>>
>>> Pretty sure you'd risk filesystem corruption by not umounting
>>> before you remove the device.

That's why I don't do that.

> This is all somewhat orthogonal to your original question though -
> these are all things that you ought to do before you remove the
> device, not after.

Quite.

> One thing I tend to do with scripts for backups to external devices
> that I don't intend to leave mounted is build the mount and umount
> into the backup script itself.

Good point.  That's probably what I should be doing instead of having
udev mount the drive when it's plugged in.  I think I'll work on that
in my spare time.  If I label the partition in question, it should be
pretty simple...

> I usually also include a check to ensure that some file/directory
> exists which I expect to be on the drive, which prevents the backup
> script from dumping a full backup into your mountpoint if it isn't
> mounted (possibly on a filesystem with insufficient space -

Yep, that's being done.  The backup won't attempt to run if the
external drive isn't mounted.

-- 
Grant Edwards               grant.b.edwards        Yow! Inside, I'm already
                                  at               SOBBING!
                              gmail.com            

Re: [gentoo-user] Re: OT: cleanup after USB backup drive unplugged?

[Wols Lists]

On 12/01/18 15:39, Grant Edwards wrote:
>> I usually also include a check to ensure that some file/directory
>> > exists which I expect to be on the drive, which prevents the backup
>> > script from dumping a full backup into your mountpoint if it isn't
>> > mounted (possibly on a filesystem with insufficient space -

> Yep, that's being done.  The backup won't attempt to run if the
> external drive isn't mounted.

Dunno quite how it works, but you could automate everything through
udev. When I stick something in, KDE offers to mount it for me, in
/run/media/$USER/abcd-efgh.

I think that last bit is unique to the media, and the same every time,
so you could make udev detect that something's been plugged in and, if
it's your backup drive, auto-run the backup.

Or, of course, if your backup expects the drive at that location, it
simply refuses to run if that location doesn't exist.

Cheers,
Wol

[gentoo-user] Re: OT: cleanup after USB backup drive unplugged?

[Grant Edwards]

On 2018-01-12, Wols Lists <antlists@youngman.org.uk> wrote:
> On 12/01/18 15:39, Grant Edwards wrote:

>>>> I usually also include a check to ensure that some file/directory
>>>> exists which I expect to be on the drive, which prevents the
>>>> backup script from dumping a full backup into your mountpoint if
>>>> it isn't mounted (possibly on a filesystem with insufficient
>>>> space -
>
>> Yep, that's being done.  The backup won't attempt to run if the
>> external drive isn't mounted.
>
> Dunno quite how it works, but you could automate everything through
> udev.

Yes, that's what I was doing.

The original question was what action should udev take when a USB
drive is unplugged.  As somebody pointed out, it's probably safer to
explicity mount the drive as part of the backup operation then unmount
when the backup is finished.  Now, instead of having udev mount the
drive when it's plugged in, the backup script handles that.

> Or, of course, if your backup expects the drive at that location, it
> simply refuses to run if that location doesn't exist.

Right.

-- 
Grant Edwards               grant.b.edwards        Yow! Is it NOUVELLE
                                  at               CUISINE when 3 olives are
                              gmail.com            struggling with a scallop
                                                   in a plate of SAUCE MORNAY?

Re: [gentoo-user] Re: OT: cleanup after USB backup drive unplugged?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

On Fri, 12 Jan 2018 16:42:25 +0000, Wols Lists wrote:

> Dunno quite how it works, but you could automate everything through
> udev. When I stick something in, KDE offers to mount it for me, in
> /run/media/$USER/abcd-efgh.
> 
> I think that last bit is unique to the media, and the same every time,
> so you could make udev detect that something's been plugged in and, if
> it's your backup drive, auto-run the backup.

Be careful with that, because udev blocks while any RUN command is
executed, so make sure whatever you run detaches immediately.


-- 
Neil Bothwick

One-seventh of life is spent on Monday.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] OT: cleanup after USB backup drive unplugged?

[Peter Humphrey]

On Friday, 12 January 2018 13:28:39 GMT Rich Freeman wrote:

> There is no need to run eject on a USB drive - just pull the thing and
> udev will clean up the device nodes.

One other small point: I've found that running eject on a USB drive on, say, 
/dev/sda marks /dev/sda unavailable for further mounts.

-- 
Regards,
Peter.

Re: [gentoo-user] OT: cleanup after USB backup drive unplugged?

[Anders Thomson]

On January 12, 2018 10:58:17 AM GMT+01:00, Adam Carter <adamcarter3@gmail.com> wrote:
>>
>> I replaced it with a USB3 drive, so I needed to update the udev rules
>> that automatically mount it and then "umount" it when it's removed.
>>
>
>Pretty sure you'd risk filesystem corruption by not umounting before
>you
>remove the device. Did it used for force an fsck on each mount because
>the
>filesystem was "dirty"? If not, i have a fundamental misunderstanding
>of
>how this stuff works.

Systemd's automount solves this nicely.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.