From: Dale <rdalek1967@gmail.com>
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] Hard drive and maximum data percentage.
Date: Sun, 2 Feb 2025 12:08:21 -0600 [thread overview]
Message-ID: <883a57f0-8bae-cbab-5c46-ee356dabef65@gmail.com> (raw)
In-Reply-To: <22583521.EfDdHjke4D@rogueboard>
Michael wrote:
> On Sunday 2 February 2025 02:07:07 Greenwich Mean Time Rich Freeman wrote:
>> On Sat, Feb 1, 2025 at 8:40 PM Dale <rdalek1967@gmail.com> wrote:
>>> Rich Freeman wrote:
>>>> Now, if you were running btrfs or cephfs or some other exotic
>>>> filesystems, then it would be a whole different matter,
>>> I could see
>>> some RAID systems having issues but not some of the more advanced file
>>> systems that are designed to handle large amounts of data.
>> Those are "RAID-like" systems, which is part of why they struggle when
>> full. Unlike traditional RAID they also don't require identical
>> drives for replication, which can also make it tricky when they start
>> to get full and finding blocks that meet the replication requirements
>> is difficult.
>>
>> With a COW approach like btrfs you also have the issue that altering
>> the metadata requires free space. To delete a file you first write
>> new metadata that deallocates the space for the file, then you update
>> the pointers to make it part of the disk metadata. Since the metadata
>> is stored in a tree, updating a leaf node requires modifying all of
>> its parents up to the root, which requires making new copies of them.
>> It isn't until the entire branch of the tree is copied that you can
>> delete the old version of it. The advantage of this approach is that
>> it is very safe, and accomplishes the equivalent of full data
>> journaling without actually having to make more than one write of
>> things. If that operation is aborted the tree just points at the old
>> metadata and the in-progress copies are inside of free space, ignored
>> by the filesystem, and thus they just get overwritten the next time
>> the operation is attempted.
>>
>> For something like ceph it isn't really much of a downside since this
>> it is intended to be professionally managed. For something like btrfs
>> it seems like more of an issue as it was intended to be a
>> general-purpose filesystem for desktops/etc, and so it would be
>> desirable to make it less likely to break when it runs low on space.
>> However, that's just one of many ways to break btrfs, so... :)
>>
>> In any case, running out of space is one of those things that becomes
>> more of an issue the more complicated the metadata gets. For
>> something simple like ext4 that just overwrites stuff in place by
>> default it isn't a big deal at all.
> I've had /var/cache/distfiles on ext4 filling up more than a dozen times,
> because I forgot to run eclean-dist and didn't get a chance to tweak
> partitions to accommodate a larger fs in time. Similarly, I've also had / on
> ext4 filling up on a number of occasions over the years. Both of my ext4
> filesystems mentioned above were created with default options. Hence -m, the
> reserved blocks % for the OS, would have been 5%. I cannot recall ever losing
> data or ending up with a corrupted fs. Removing some file(s) to create empty
> space allowed the file which didn't fit before to be written successfully and
> that was that. Resuming whatever process was stopped (typically emerge)
> allowed it to complete.
>
> I also had smaller single btrfs partitions binding up a couple of times. I
> didn't lose any data, but then again these were stand alone filesystems not
> part of some ill advised buggy btrfs RAID5 configuration.
>
> I don't deal with data volumes of the size Dale is playing with, so I can't
> comment on suitability of different filesystems for such a use case.
>
This is all good info. It's funny, I think me using ext4 was the best
thing for this situation. It works well for storing all the files I
have plus I can fill it up pretty good if I have too. Thing is, I could
stop the download of some videos if needed. At least until I can get a
new drive to expand with.
I'll be getting another drive for Crypt next. Then I should be good for
a while.
Dale
:-) :-)
prev parent reply other threads:[~2025-02-02 18:09 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-01-26 18:15 [gentoo-user] Hard drive and maximum data percentage Dale
2025-02-01 21:15 ` Dale
2025-02-01 22:10 ` Mark Knecht
2025-02-01 23:51 ` Dale
2025-02-01 21:55 ` Rich Freeman
2025-02-02 0:15 ` Dale
2025-02-02 0:29 ` Rich Freeman
2025-02-02 1:40 ` Dale
2025-02-02 2:07 ` Rich Freeman
2025-02-02 11:00 ` Michael
2025-02-02 18:08 ` Dale [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=883a57f0-8bae-cbab-5c46-ee356dabef65@gmail.com \
--to=rdalek1967@gmail.com \
--cc=gentoo-user@lists.gentoo.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox