Re: [gentoo-user] Organising btrfs subvolumes

[Rich Freeman <rich0@gentoo.org>]

On Tue, May 27, 2014 at 7:49 AM, Neil Bothwick <neil@digimed.co.uk> wrote:
> I have zfs-snapshot making snapshots at 15 minute, hourly, daily, monthly
> and weekly intervals - and it cleans up after itself. There isn't
> anything quite like that for btrfs, so I'm knocking up a python script to
> take care of it. I want automated snapshots before I risk it on my
> desktop.

There is snapper, which is even in the tree now.  It isn't 100%
flexible but supports any number of hourly, daily, monthly, and yearly
snapshots, with retention policies for each.

My problem was that the snapshots were created hourly, but cleaned up
daily, which meant 24 deleted in parallel at a time.

In general I've tended to notice that btrfs tends to suffer from
hurry-up-and-wait issues.  It will happily accept a ton of writes
(even from an ioniced process) which I imagine go into the log, and
then the whole filesystem will come to a halt at the next checkpoint
(every 30s by default).  It makes ionice just about useless, since the
filesystem accepts more data than it can handle, and then blocks even
realtime-priority processes while it is catching up.

I suspect that it was having a related issue with snapshot removals.
24 huge snapshot removal commands complete in almost zero time, and
then in 30s the debt comes due.

In order to be a bit more ionice-friendly the filesystem needs to
figure out what it can sustain and throttle writes to a reasonable
rate.  I'm fine with having some allowance for bursting, but having
all disk access block for 10-20 seconds isn't acceptable.

Oh, and chromium just loves its disk cache - it will happily wait for
20 seconds to read a cache entry that it could have downloaded online
in less than a second.

Rich