[gentoo-user] btrfs and sparse VM image files

[gentoo-user] btrfs and sparse VM image files

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 489 bytes --]

I'm confused about the desirability of keeping VM image files, usually
space qcow files, on a btrfs volume. I have read the advice about using
chattr +C on the subvolume, but are there any other gotchas? The btrfs
wiki says in one place that using sparse file on btrfs is not a good
idea, but is that still the case. There is conflicting information out
there, does anyone here have any hard experience?


-- 
Neil Bothwick

Beware of the opinion of someone without any facts.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] btrfs and sparse VM image files

[Stefan G. Weichinger]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 18.05.2014 14:28, schrieb Neil Bothwick:
> I'm confused about the desirability of keeping VM image files,
> usually space qcow files, on a btrfs volume. I have read the advice
> about using chattr +C on the subvolume, but are there any other
> gotchas? The btrfs wiki says in one place that using sparse file on
> btrfs is not a good idea, but is that still the case. There is
> conflicting information out there, does anyone here have any hard
> experience?

No clue yet, but interested as well.
Although I avoid these spaced qcow-files anyway.

I would like to run QEMU/KVM-based VMs on btrfs subvolumes ... I love
the idea of snapshotting them. Until now I often run them on LVM-LVs
(as "raw") and use virt-backup (which uses LVM snapshots) to generate
backups.

That works OK but btrfs-snapshots look and are way more sophisticated,
I assume.

- -

I currently move over data from my zfs-pool to a new btrfs pool and
will remove zfs then from that server.

Stefan


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTearvAAoJEClcuD1V0Pzm3pcQAIq2yN5/2b8iFeLOBeY8Omwo
JykaBvY8oSTa1julDs0JxKeEl7c4BKKB2lqUCpQ1Sl0G4TY4UR3stK/kKbA2X4Y5
9uJsh91U7yz/KsX3r+4d+e8B8iazhT66zY1Xyk94xqMQXjgpOrdDIMbfaK/aWeda
4pz5R47LNLSCEBhNCg0uZV/iv+oXTXGpvtnQ1B/bY15cf122Z59XHDZT9ofEK9EM
zS4IucVHQpHS9OXmeusKqYmuos1s/WVYBmxfLJnfiMrd1llbMS9L63K2QV1YWp37
KSimPhyijpTIGPPdUMGjNaVzE2tE3cGOT3j0r33BVap62az1ese4Bi8fEES09bOk
MjqGRnki1ZBjOx5cOCDmawvkBgMJzFZvqLofoCi0qGTegk33cXY5PBpPrXVfG2Qn
7WaVifl+oR16mVvpMKYUfoVmRZz93X8ZWQQT0Rfxtw3Sr+k9U6RUYl8lb/KqArAv
kSKFH3jqRKpXJ7bfevKdB7YZp4+pKKf4PR3AHHLSkYJbM6H/1His1YNfdAfn074G
3D/HSIJNpojBRybpwM4eeYcB9DxqG712LkjNGHyvnfAuUpsnjUVQrBPLhblajvDy
J7K0PdfYPUm37Hl4cVk3ohlM4h7kIxLDxptmV6darB1JAQV+J/G5R6HqRxfdvOHN
s75mo/Ph+pKG/H4SM1Uk
=wWUR
-----END PGP SIGNATURE-----

Re: [gentoo-user] btrfs and sparse VM image files

[Marc Stürmer]

Am 18.05.2014 14:28, schrieb Neil Bothwick:

> I'm confused about the desirability of keeping VM image files, usually
> space qcow files, on a btrfs volume. I have read the advice about using
> chattr +C on the subvolume, but are there any other gotchas? The btrfs
> wiki says in one place that using sparse file on btrfs is not a good
> idea, but is that still the case. There is conflicting information out
> there, does anyone here have any hard experience?

Just take a look at the official Gotchas Page of BTRFS, which can be 
found here:

https://btrfs.wiki.kernel.org/index.php/Gotchas

Putting virtual image files on Btrfs is something that the developers 
still do not recommend at all, and that's with reason!

If you really do want to put them up a COW filesystem, you should try 
ZFS on Linux instead, otherwise go with XFS or ext4 - in that kind of 
order.

Frankly said, Btrfs in my humble opinion is just not ready for prime 
time yet and will not be for a couple of year and if you really do want 
a COW filesystem now, you should take a look at ZFS instead.

Re: [gentoo-user] btrfs and sparse VM image files

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 958 bytes --]

On Mon, 19 May 2014 12:07:32 +0200, Marc Stürmer wrote:

> Just take a look at the official Gotchas Page of BTRFS, which can be 
> found here:
> 
> https://btrfs.wiki.kernel.org/index.php/Gotchas
> 
> Putting virtual image files on Btrfs is something that the developers 
> still do not recommend at all, and that's with reason!

The page you linked to does not actually state that. There are plenty of
hints and sideways references but little concrete information about what
is safe with the current release - hence my question.

> If you really do want to put them up a COW filesystem, you should try 
> ZFS on Linux instead, otherwise go with XFS or ext4 - in that kind of 
> order.

They are already on ZFS but I am investigating btrfs as an alternative to
ZFS. ZFS and ext4 would mean losing the volume management that ZFS and
btrfs offer, not to mention forcing a repartition.


-- 
Neil Bothwick

Do you steal taglines too?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] btrfs and sparse VM image files

[Rich Freeman]

On Mon, May 19, 2014 at 7:01 AM, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Mon, 19 May 2014 12:07:32 +0200, Marc Stürmer wrote:
>
>> Just take a look at the official Gotchas Page of BTRFS, which can be
>> found here:
>>
>> https://btrfs.wiki.kernel.org/index.php/Gotchas
>>
>> Putting virtual image files on Btrfs is something that the developers
>> still do not recommend at all, and that's with reason!
>
> The page you linked to does not actually state that. There are plenty of
> hints and sideways references but little concrete information about what
> is safe with the current release - hence my question.

I haven't had significant issues with casually running VMs on btrfs,
but right now I wouldn't say the performance is spectacular.  I do
have my VM images set to use COW - I'd rather take a performance hit
than not have data protected.  If performance were a big concern I'd
probably end up setting up an ext4 on mdadm+lvm, but I really don't
want to go splitting up my drives as managing that was a real pain in
the past (mdadm is much less flexible than btrfs when you have drives
of differing sizes).

>
>> If you really do want to put them up a COW filesystem, you should try
>> ZFS on Linux instead, otherwise go with XFS or ext4 - in that kind of
>> order.
>
> They are already on ZFS but I am investigating btrfs as an alternative to
> ZFS. ZFS and ext4 would mean losing the volume management that ZFS and
> btrfs offer, not to mention forcing a repartition.

How does ZFS prevent fragmentation?  Does it use COW for all writes (I
thought it did)?  The fundamental issue is that data is never
overwritten in place. That means that if you change one block in a 2GB
file, you end up with two extents for the file, until things get bad
enough that the OS ends up copying the entire file into a single
extent.  Maybe another strategy (if there aren't any impacted
snapshots) is to overwrite data in place using a journal when you have
a file with many random writes (basically like journal=data mode on
ext4).  That would be a bit like creating a second extent and then
when there is time moving it back on top of the first extent.

Once you have a snapshot I'd think you'd never be able to prevent
fragmentation, though I guess if you're clever you could merge extents
that share common snapshots.

Has ZFS actually been shown to perform well for VMs in comparison to
ext4?  If so, I wonder how they do it.

Rich

Re: [gentoo-user] btrfs and sparse VM image files

[Stefan G. Weichinger]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am 19.05.2014 13:01, schrieb Neil Bothwick:
> On Mon, 19 May 2014 12:07:32 +0200, Marc Stürmer wrote:
> 
>> Just take a look at the official Gotchas Page of BTRFS, which can
>> be found here:
>> 
>> https://btrfs.wiki.kernel.org/index.php/Gotchas
>> 
>> Putting virtual image files on Btrfs is something that the
>> developers still do not recommend at all, and that's with
>> reason!
> 
> The page you linked to does not actually state that. There are
> plenty of hints and sideways references but little concrete
> information about what is safe with the current release - hence my
> question.

I have a nice testbox here now ... a server for a customer which I
won't deliver soon so I will have that one here for some weeks.

I will play around with qemu/kvm-vms in various configs ... and see
what happens.

Stefan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTewhlAAoJEClcuD1V0PzmQFUP/AzzW5ilQpK+RvWsikTWvyX7
LxGQQ9dTnCYNzXzLsf9drPDMAgQiIppJ/T3INRrsqCfCgpiIdOIx5YEMCW4at61m
BbeXtUR6RazAcgI+2hOqbnkZFCVyOwG3Od2+C87y1NWc4+EId2ryNpo+eOlqC7Zr
xlyqs2gE7HclVC29NQx2ntw19y9JMJD/Ac0I7oL5lKZt45bywBHzlcXplmuSDhba
xuJuNT6ei+ip3A2+bYrGvfQ5/Yd0TiGUtbpjkAWPR1WXGdszqhvpG73aGlpj9aff
gU6phiZc0vLjtGuM5peDSbNoT5OGpQJdhUTU+DbEpCdq1yh19h03Xa8YUibSFUT/
IZaB4UyjIr8Y/6SKF1krcLIrCi+1vvgRw2D2JscgLw/5uQ5sPX3LUay4gFxtWafq
IHLbFvvjtrra847nGnNOKf9iUbbSv96lUXmvqJx+2sxr2ESsHqHpTFwAYWtRKz41
2A2cufUC6Zx7G/mNsE1teajovCP48EewlF3UKFo0Tqc68iuLMdQ4RJ4wVX5J12sk
Ma0K/Y28/qezTO9GrJYzo4UoFxzE5md9foV0xqS34PMNtiw8aYNK3pC83AOE9KBu
+daOIH6EY7HWx/C8S7jzeQZu8pWKOGf2rIT65Xt1Ij1w7vBXgD/4B5DeUkacAnz4
P/2sbXDzqrZr8PO32AOW
=+l8k
-----END PGP SIGNATURE-----

Re: [gentoo-user] btrfs and sparse VM image files

[Marc Stürmer]

Am 19.05.2014 13:01, schrieb Neil Bothwick:

> The page you linked to does not actually state that. There are plenty of
> hints and sideways references but little concrete information about what
> is safe with the current release - hence my question.

Oh it does, just take a look at that section:

-----
Files with a lot of random writes can become heavily fragmented (10000+ 
extents) causing trashing on HDDs and excessive multi-second spikes of 
CPU load on systems with an SSD or large amount a RAM.

* On servers and workstations this affects databases and virtual
                                                          ^^^^^^^
machine images.
^^^^^^^^^^^^^^^

* The nodatacow mount option may be of use here, with associated gotchas.
-----

So they still do not recommend putting virtual machine images on a Btrfs 
(if you want it in productional use, that is).