From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 5254915ACFB for ; Wed, 19 Apr 2023 23:32:53 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id BAA18E088F; Wed, 19 Apr 2023 23:32:48 +0000 (UTC) Received: from mail-yw1-x1134.google.com (mail-yw1-x1134.google.com [IPv6:2607:f8b0:4864:20::1134]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 33D6FE0884 for ; Wed, 19 Apr 2023 23:32:48 +0000 (UTC) Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-54fe3cd445aso21782367b3.5 for ; Wed, 19 Apr 2023 16:32:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681947167; x=1684539167; h=in-reply-to:mime-version:user-agent:date:message-id:autocrypt :openpgp:from:references:to:subject:from:to:cc:subject:date :message-id:reply-to; bh=kCZvqo0kfoYZgmdJNbXtvvrn8r2pHpltxGG182a/J3A=; b=EEMIcjmnKW2xpGnaHEYSNNzthASEK34/JVDRx4Gq6o5Ma4BLdbDAggvTWHPjrSKEma lKhVkLki1scb9c6Mj/nVC5P2JgyWl6FBQyixq7N2m7DFfhSKa+aehMDC4bt6MWqwqEdp 1swWIxxx2FsiPM0NyZdmz7z4Pmr/LHQhGbpOSFFe0bv3UagVb27ZF2m7DtciCzpUfZJS 9I77A2JOIDC/UQd77tRSB9dqyLweilLYZm+lGdAws4ijWbqYgtGWHWJNyCBCFIc1WyCL MkbDGzNXwVo7ZKP7KtePzEM/ciZQf4ClrI9haNXGxlUBIpFYAr8VopmQ0dMdHUGxJcNB Zdkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681947167; x=1684539167; h=in-reply-to:mime-version:user-agent:date:message-id:autocrypt :openpgp:from:references:to:subject:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kCZvqo0kfoYZgmdJNbXtvvrn8r2pHpltxGG182a/J3A=; b=KPymphpXY1Kq0FlPFz913ET+JP44XoAo96rC2wTqHu/+6fngxDvzZrkGSSgycl3mny xydDgtlwQeSr65b+i/dybrYb50ZNpqTTzQmzgtl3p/tdTGKZJVp1BeT0EDcl1s8P/NYd Wvl1YQVrrVRoxrzN57TXr4CFbQjDjqAvJ35qzEAAXBKJ1u4BUWGq4SXJLq34sQnDVQaU jPD2dW3bLFLWWkNifwUCCRrJhhLEy6k/+JommQ1ZKF5MDa63YHLIw24QM300GT0Om4tz 2dxZPPNvOMrI2axCQqBCay6q5r4zi9gbGtDavuo7n9iEXvgQrtIZ9qJGX7qluZVzlMST waTQ== X-Gm-Message-State: AAQBX9eHsb1+nemvQEX5/ZXq5pdSI/jeZcPc8V2y85F6aEEE0QuWrMqO D+0m31x3BjfpbkPNDQ7O90kjL8iJsdQEjA== X-Google-Smtp-Source: AKy350Y5RRLMdqGk/sP0TgtuYbG1wmB9xcDT5M4eGAw6LyKj6MedgP6i8aciF7JPzLxhf2muu/Adjw== X-Received: by 2002:a81:6541:0:b0:54f:b4eb:bc14 with SMTP id z62-20020a816541000000b0054fb4ebbc14mr5501216ywb.3.1681947167402; Wed, 19 Apr 2023 16:32:47 -0700 (PDT) Received: from [10.8.8.5] ([107.179.20.172]) by smtp.gmail.com with ESMTPSA id r2-20020a815d02000000b0054f97b52934sm4589ywb.54.2023.04.19.16.32.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 19 Apr 2023 16:32:47 -0700 (PDT) Subject: Re: [gentoo-user] Re: Finally got a SSD drive to put my OS on To: gentoo-user@lists.gentoo.org References: <3a8a143d-38f0-b7ea-4aa1-10c0b3a2a1e0@gmail.com> <81fde7b8-cc55-fcae-79ea-4dd2a78eb8a0@gmail.com> <2703063.mvXUDI8C0e@wstn> <7ef36857-192e-6d2c-dc44-a2c4ba772205@gmail.com> <35474652-f973-75b2-b8e7-4d22915672ac@gmail.com> From: Dale Openpgp: preference=signencrypt Autocrypt: addr=rdalek1967@gmail.com; prefer-encrypt=mutual; keydata= mQINBGFSciYBEADcEGMyJBSuavKO/XKUVvgkxck7Nl8Iuu8N2lcnRji/rSKg5c1Acix1ll9i oW8JBCHwvn0+Xy60BvEsqcup3YSHw5STl/bR1ePEehtnYrg8FdjdS91+B805RfnKMm69rFVI wLSBHQrSG1yxHd8CloWoEdhmVtP24buajbh114bgXd9ahtpZrCVMrWdWYUg2mEXguGV5uNAh Rf8SWxDNc79w24JxsV34a8niMUYMjzWr0rafIbzk732X38vGjVMLo/2mMpkbp9mPp++LHoY+ 0Pet8zxxdXPJSCd475kza1AD+hhSyBZXB9yknYWgyY3cZe1rGmooJSi2KX4QxO7npwLThcO1 be6KKRkd35+Fi/a1BzVOHsZMiK/gcwxEFoMd27gir4ehaeHJfFXl+65w4hj0EsOZSxrJrm2C R50g5By2czSKP1bADEygFNpIJj51AR+wM88NImG2RPtlT2maYBzazvF05g65cdHXGp1C7W5P wwwKU2DgABB2t7N7z5A69LnryBRw4zUYDRRYLTYlBlYgg+xILm2c0OrBdxJgLJa7JE50Eo25 d3PFwt9J0gYvqy6sPFLl9So0sDg9zm0hKQtXOP5kgropUFGrNoJI+mjwF4rYLRBVzZwNAvlO OhEvHubBo3mEllv4x+FeptwXZxlk7gUsdqI8AxnFB8K9wi6FVQARAQABtBtEYWxlIDxyZGFs ZWsxOTY3QGdtYWlsLmNvbT6JAk4EEwEIADgCGyMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AW IQQSG1h01ruv/WNXc3Q3RqOgiQH1GwUCYVJy8gAKCRA3RqOgiQH1G+waEACeTZCt77jnRAmQ AV7otKuZekDWiLi3Eig8tj5ZJiCNSYA/hIxzmexRP0GMqjitcXK1iGwWcvMzzvIq30GAjIfB 4BR38cnXbtBa6fNewiT7QaZe/Hn6yBRldXNQypzbHy+/o27bUEy+oX4rE7etUgEHQAjuw7xz XFWg4tH1/KJvsOVY5upnWc5LdxYhsuQ3dQD4b22GsK0pOBDfb9PiirYM8eGKvrVuq4E/c75z lDDFhINl18lNZ9D0ZFL3IkTjHsAAqFH9uhnnEB8CWdHbBewPEfRaOhBUYWZ3Q8uTkmDgZT8q D9jlvLEdw7Nh2ApdxoepnI/4D+ql2Gr4DtH7SEPydr5gcf1Qr/2bXRb1hAYnIVcbncs/Bm3Z bkRKPVWMfE3Fusa+p5hMzixk0YysMaTHlc7mYRYAEZGnPMXnmcCbetwARU7A0yz1M1kCMOAQ Lsz8KH5kv3cRenMB6SFfjND2JfAK61H5TtnPq3L8noS2ZykRYxq9Nm3X64O1tJojIKBoZFr8 AwYNCvqC6puUyGMuzHPh7jPof8glfrrEKIYUvNPGMDoVX3IGetxh/9l6NcxgFA4JGoR+LS3C zmeNrwlllAe3OEUfKoWVQ+pagpSdM+8hHolaSda4Ys66Z3fCR4ZvcTqfhTAVskpqdXa4isAk 7vTcXu3L499ttywEp7rJTbkCDQRhUnImARAAncUdVhmtRr59zqpTUppKroQYlzR0jv8oa7DG K4gakTAT2N7evnI9wpssmzyVk8VEiLzhnFQ/Ol3FRt6hZCXDJt0clyHOyTfvz/MNFttWuZTc mLpSvmRR6VRjAH+Tz3Eam2xUw3PGuH97BcXQ3NnX3msv1UDxtxxBu6e2YrdeOhrCUSgzokcJ 98ChUNy934cgepPybAI12lSWqVFQ1aG7jExZfiUk+333fPSDbpKoZbTW5YJLXbycmW/C1IWL qYQyNjRWKaGoJtUWFhhmNiOQct7n90aKivNVPavmN+UQ9LlMaINtf9T6XCzLfogCFsulDCDJ 0yNQLDTurHaB4E71xoctgXmLLq9z1RQ0W2XiVAAOZQj6K3+d0AOUjDhCQ2QW8dUSq0ckkZXV DKVJOGS8Nhf2eIWIqRnP3AcUiiaiFGqUaVUmUAZ6h/oJmgghEu/1S+pcuUKU5i69+XCZ3hH2 Jzwzbf7K+FAIkOhCfHncF8i1N1pk00pOVykNnqHTfFo3qFusHt0ZWgXVnnn4pYdXqZNoDhvF BRE5Vm4k/k96Pw8HRx6Os6eFSRrlqGzRgqsu86FekxusXB9UGv4lJhtU/J+8MRWsh22K718s DbQnABicGKFz1qQlWvcf59oTByhLINJCBt1WXl+TzJDXepr3QSkqmK41dO9Hob97C9dMiK8A EQEAAYkCNgQYAQgAIAIbDBYhBBIbWHTWu6/9Y1dzdDdGo6CJAfUbBQJhUnLyAAoJEDdGo6CJ AfUbVHIQAKSWw620vPhR3A/njU2z77F3z/Jk+HTKdE3fIyWSWdkYN7CBFL0NguOMP30WZ+qE sJhZu7T5hf251MwQUUt27xlfnKYOmQs7CqONlXuXlGZI6WufrUjxNcVz+5gJsqvUWuuJWsgg sDmE92IBnfG/f81fPHWQyfr/SF4wYDMyoFp5xCCQpp1zB63iuFvvrhxBkEHzmbRtVDOhl0Xp BVEDR1w3QRACw9QJD/KM05Czv9JNQYlwinWO/OaQ9cMlUpKLgswUPg9IZ5vucxScfuAUA5uC B1jlAQ8ZPlVukBmbEv5RGOv+lpuEbA3YDMVtEeH4YMFbjt/+vH3Cr2vTbp5JlpByLburJEH0 WXZLUawEfUsZvVwpOuJK75vaa2HYXee+Cb3iCIzwfIfctdlqzUcbGRczlRNM59hpvj4z29Gh 3kAxVHItAYq54ikxQ9l4hQ8s9sLYPbX/WtcBxNX8crBSw0FLnmzGleVEtBHyqtt5CLzQNgrj GYWl1vKDUmRPw1CdZ1c+fMN9CY11jOM5B5ZnqZWfDeVYO2iJ5SuvTycChexCb8WYn1bdCBIo bBtga2RBXbVt4Mh9E4owsszefn51MwfjXxB20Fc5k3GU1AVpTCMs3ayYCzo0b2pvEvdjtDcA CYLEFPWgaFX9iQAM/CDfKvTtvgGWpqtCL2raq/mQoJEU Message-ID: <9781e250-591d-d468-11f2-7e5c94ac7db3@gmail.com> Date: Wed, 19 Apr 2023 18:32:45 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0 SeaMonkey/2.53.15 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------EA96616F5DA26F626C4D08A1" X-Archives-Salt: 0df1e245-1efa-45e5-8eae-1e3d155ceab2 X-Archives-Hash: bffa36b7f12ffc8f78079284fb087743 This is a multi-part message in MIME format. --------------EA96616F5DA26F626C4D08A1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Frank Steinmetzger wrote: > <<>> > > When formatting file systems, I usually lower the number of inodes from the > default value to gain storage space. The default is one inode per 16 kB of > FS size, which gives you 60 million inodes per TB. In practice, even one > million per TB would be overkill in a use case like Dale’s media storage.¹ > Removing 59 million inodes × 256 bytes ≈ 15 GB of net space for each TB, not > counting extra control metadata and ext4 redundancies. > > The defaults are set in /etc/mke2fs.conf. It also contains some alternative > values of bytes-per-inode for certain usage types. The type largefile > allocates one inode per 1 MB, giving you 1 million inodes per TB of space. > Since ext4 is much more efficient with inodes than ext3, it is even content > with 4 MB per inode (type largefile4), giving you 250 k inodes per TB. > > For root partitions, I tend to allocate 1 million inodes, maybe some more > for a full Gentoo-based desktop due to the portage tree’s sheer number of > small files. My Surface Go’s root (Arch linux, KDE and some texlive) uses > 500 k right now. > > > ¹ Assuming one inode equals one directory or unfragmented file on ext4. > I’m not sure what the allocation size limit for one inode is, but it is > *very* large. Ext3 had a rather low limit, which is why it was so slow with > big files. But that was one of the big improvements in ext4’s extended > inodes, at the cost of double inode size to house the required metadata. > This is interesting.  I have been buying 16TB drives recently.  After all, with this fiber connection and me using torrents, I can fill up a drive pretty fast, but I am slowing down as I'm no longer needing to find more stuff to download.  Even 10GB per TB can add up.  For a 16TB drive, that's 160GBs at least.  That's quite a few videos.  I didn't realize it added up that fast.  Percentage wise it isn't a lot but given the size of the drives, it does add up quick.  If I ever rearrange my drives again and can change the file system, I may reduce the inodes at least on the ones I only have large files on.  Still tho, given I use LVM and all, maybe that isn't a great idea.  As I add drives with LVM, I assume it increases the inodes as well.  If so, then reducing inodes should be OK.  If not, I may increase drives until it has so many large files it still runs out of inodes.  I suspect it adds inodes when I expand the file system tho and I can adjust without worrying about it.  I just have to set it when I first create the file system I guess. This is my current drive setup.  root@fireball / # pvs -O vg_name   PV         VG     Fmt  Attr PSize    PFree   /dev/sda7  OS     lvm2 a--  <124.46g 21.39g   /dev/sdf1  backup lvm2 a--   698.63g     0   /dev/sde1  crypt  lvm2 a--    14.55t     0   /dev/sdb1  crypt  lvm2 a--    14.55t     0   /dev/sdh1  datavg lvm2 a--    12.73t     0   /dev/sdc1  datavg lvm2 a--    <9.10t     0   /dev/sdi1  home   lvm2 a--    <7.28t     0 root@fireball / # The one marked crypt is the one that is mostly large video files.  The one marked datavg is where I store torrents.  Let's not delve to deep into that tho.  ;-)  As you can see, crypt has two 16TB drives now and I'm about 90% full.  I plan to expand next month if possible.  It'll be another 16TB drive when I do.  So, that will be three 16TB drives.  About 43TBs.  Little math, 430GB of space for inodes.  That added up quick.  I wonder.  Is there a way to find out the smallest size file in a directory or sub directory, largest files, then maybe a average file size???  I thought about du but given the number of files I have here, it would be a really HUGE list of files.  Could take hours or more too.  This is what KDE properties shows. 26.1 TiB (28,700,020,905,777) 55,619 files, 1,145 sub-folders Little math. Average file size is 460MBs. So, I wonder what all could be changed and not risk anything??? I wonder if that is accurate enough??? Interesting info. Dale :-) :-) --------------EA96616F5DA26F626C4D08A1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
Frank Steinmetzger wrote:
<<<SNIP>>> 

When formatting file systems, I usually lower the number of inodes from the 
default value to gain storage space. The default is one inode per 16 kB of 
FS size, which gives you 60 million inodes per TB. In practice, even one 
million per TB would be overkill in a use case like Dale’s media storage.¹ 
Removing 59 million inodes × 256 bytes ≈ 15 GB of net space for each TB, not 
counting extra control metadata and ext4 redundancies.

The defaults are set in /etc/mke2fs.conf. It also contains some alternative 
values of bytes-per-inode for certain usage types. The type largefile 
allocates one inode per 1 MB, giving you 1 million inodes per TB of space. 
Since ext4 is much more efficient with inodes than ext3, it is even content 
with 4 MB per inode (type largefile4), giving you 250 k inodes per TB.

For root partitions, I tend to allocate 1 million inodes, maybe some more 
for a full Gentoo-based desktop due to the portage tree’s sheer number of 
small files. My Surface Go’s root (Arch linux, KDE and some texlive) uses 
500 k right now.


¹ Assuming one inode equals one directory or unfragmented file on ext4.
I’m not sure what the allocation size limit for one inode is, but it is 
*very* large. Ext3 had a rather low limit, which is why it was so slow with 
big files. But that was one of the big improvements in ext4’s extended 
inodes, at the cost of double inode size to house the required metadata.


This is interesting.  I have been buying 16TB drives recently.  After all, with this fiber connection and me using torrents, I can fill up a drive pretty fast, but I am slowing down as I'm no longer needing to find more stuff to download.  Even 10GB per TB can add up.  For a 16TB drive, that's 160GBs at least.  That's quite a few videos.  I didn't realize it added up that fast.  Percentage wise it isn't a lot but given the size of the drives, it does add up quick.  If I ever rearrange my drives again and can change the file system, I may reduce the inodes at least on the ones I only have large files on.  Still tho, given I use LVM and all, maybe that isn't a great idea.  As I add drives with LVM, I assume it increases the inodes as well.  If so, then reducing inodes should be OK.  If not, I may increase drives until it has so many large files it still runs out of inodes.  I suspect it adds inodes when I expand the file system tho and I can adjust without worrying about it.  I just have to set it when I first create the file system I guess.

This is my current drive setup. 


root@fireball / # pvs -O vg_name
  PV         VG     Fmt  Attr PSize    PFree
  /dev/sda7  OS     lvm2 a--  <124.46g 21.39g
  /dev/sdf1  backup lvm2 a--   698.63g     0
  /dev/sde1  crypt  lvm2 a--    14.55t     0
  /dev/sdb1  crypt  lvm2 a--    14.55t     0
  /dev/sdh1  datavg lvm2 a--    12.73t     0
  /dev/sdc1  datavg lvm2 a--    <9.10t     0
  /dev/sdi1  home   lvm2 a--    <7.28t     0
root@fireball / #


The one marked crypt is the one that is mostly large video files.  The one marked datavg is where I store torrents.  Let's not delve to deep into that tho.  ;-)  As you can see, crypt has two 16TB drives now and I'm about 90% full.  I plan to expand next month if possible.  It'll be another 16TB drive when I do.  So, that will be three 16TB drives.  About 43TBs.  Little math, 430GB of space for inodes.  That added up quick. 

I wonder.  Is there a way to find out the smallest size file in a directory or sub directory, largest files, then maybe a average file size???  I thought about du but given the number of files I have here, it would be a really HUGE list of files.  Could take hours or more too.  This is what KDE properties shows.

26.1 TiB (28,700,020,905,777)

55,619 files, 1,145 sub-folders

Little math. Average file size is 460MBs. So, I wonder what all could be changed and not risk anything??? I wonder if that is accurate enough???

Interesting info.

Dale

:-) :-)

--------------EA96616F5DA26F626C4D08A1--