From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 897C7158042 for ; Fri, 15 Nov 2024 01:00:16 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id D2823E096C; Fri, 15 Nov 2024 01:00:09 +0000 (UTC) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 85B60E0951 for ; Fri, 15 Nov 2024 01:00:09 +0000 (UTC) Received: by mail-io1-f45.google.com with SMTP id ca18e2360f4ac-83b430a4cfdso43961739f.2 for ; Thu, 14 Nov 2024 17:00:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731632408; x=1732237208; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SV9g3zqydYidH5stC75U6RkvauPUhuC7+0tjeifC+Zc=; b=gyx5Kck0lR2+og+GKlAyNZN2Ny/EiZn0QtLZjLxKB+aeKXlccI4Mggry3zFIImmgMP 0xiqzXBSGEvPgapHqUtMNlFxBjWmpNko748T2rbQH39eAttNwxb2r5nxuPC2tUbh4WqS +xXRNSSp0+sbxIAS76SKTDqJk97YDSxT+/GKGvcU+X26Er5zsnTled1Ntuly22EBf+Up MAW1hhgCe67jn4labeLlHwMBeBzVk3Wj51Rr0DOfc4qOXqDU46APprQxyfQgtRm4pZUV XQDE6Xv81bpF67fsQo9GzKRObkPyKV+jl1w1HDoYuRgtuxZ+63p4VSmOaNGJEOyHaRY/ WrqQ== X-Gm-Message-State: AOJu0Yyvje9FYhCtfsmOSV37KXZRN6Bmag6VmpUS/dpzmmLVUdt2uzzM HZssdcbbfIJnMrcvEOGZu5xx+2+KU0upxPMwX5fjNuc7IZ6rr7FiRctRbhCUDJNf90ucEtsvPHs BIjvx49slhJOzPDLnfkCpLUwBKKDub67p X-Google-Smtp-Source: AGHT+IGXPRhRQGOCVYewEhC23TN05BXR98dlEroB8PlHcaUb4opNbXFM6GOvUWnBWq6dqLAU3lHYtgSeT92hegUhx/A= X-Received: by 2002:a05:6602:1507:b0:83a:c4e1:7d69 with SMTP id ca18e2360f4ac-83e6c26442emr102859339f.1.1731632408285; Thu, 14 Nov 2024 17:00:08 -0800 (PST) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 References: <1879585.atdPhlSkOF@rogueboard> <5835905.DvuYhMxLoT@rogueboard> <6461ae0f-3a98-6325-9915-3c78b0df237a@gmail.com> <63e3fac6-1758-b5c2-2c92-312d64d547e3@gmail.com> In-Reply-To: <63e3fac6-1758-b5c2-2c92-312d64d547e3@gmail.com> From: Rich Freeman Date: Thu, 14 Nov 2024 19:59:57 -0500 Message-ID: Subject: Re: [gentoo-user] Seagate hard drives with dual actuators. To: gentoo-user@lists.gentoo.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Archives-Salt: 15c3662f-dfd8-4751-8929-e616a0f98b92 X-Archives-Hash: 0544c941fd1c720d3110c20ba3e95727 On Thu, Nov 14, 2024 at 6:10=E2=80=AFPM Dale wrote: > > The biggest downside to the large drives available now, even if SMART > tells you a drive is failing, you likely won't have time to copy the > data over to a new drive before it fails. On a 18TB drive, using > pvmove, it can take a long time to move data. Very true. This is why I'm essentially running RAID6. Well, that and for various reasons you don't want to allow writes to Ceph without at least one drive worth of redundancy, so having an extra replica means that you can lose one and remain read-write, and then if you lose a second during recovery you might be read-only but you still have data integrity. (Don't want to get into details - it is a Ceph-specific issue.) > I don't even want to think what it would cost to put > all my 100TBs or so on SSD or NVME drives. WOW!!! # kubectl rook-ceph ceph osd df class ssd ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS 8 ssd 6.98630 1.00000 7.0 TiB 1.7 TiB 1.7 TiB 63 MiB 3.9 GiB 5.3 TiB 24.66 1.04 179 up 4 ssd 1.74660 1.00000 1.7 TiB 465 GiB 462 GiB 16 MiB 2.5 GiB 1.3 TiB 25.99 1.10 45 up 12 ssd 1.74660 1.00000 1.7 TiB 547 GiB 545 GiB 30 MiB 2.1 GiB 1.2 TiB 30.57 1.29 52 up 1 ssd 6.98630 1.00000 7.0 TiB 1.7 TiB 1.7 TiB 50 MiB 4.2 GiB 5.3 TiB 24.42 1.03 177 up 5 ssd 6.98630 1.00000 7.0 TiB 1.8 TiB 1.8 TiB 24 MiB 5.0 GiB 5.2 TiB 25.14 1.07 180 up 3 ssd 1.74660 1.00000 1.7 TiB 585 GiB 583 GiB 18 MiB 2.0 GiB 1.2 TiB 32.70 1.39 57 up 21 ssd 1.74660 1.00000 1.7 TiB 470 GiB 468 GiB 27 MiB 1.9 GiB 1.3 TiB 26.26 1.11 52 up 9 ssd 1.74660 1.00000 1.7 TiB 506 GiB 504 GiB 11 MiB 2.0 GiB 1.3 TiB 28.29 1.20 49 up 18 ssd 1.74660 1.00000 1.7 TiB 565 GiB 563 GiB 16 MiB 1.7 GiB 1.2 TiB 31.59 1.34 55 up 10 ssd 1.74660 1.00000 1.7 TiB 490 GiB 489 GiB 28 MiB 1.6 GiB 1.3 TiB 27.42 1.16 53 up 22 ssd 1.74660 1.00000 1.7 TiB 479 GiB 478 GiB 19 MiB 1.7 GiB 1.3 TiB 26.80 1.14 50 up 19 ssd 13.97249 1.00000 14 TiB 2.3 TiB 2.3 TiB 87 MiB 5.2 GiB 12 TiB 16.81 0.71 262 up TOTAL 49 TiB 12 TiB 12 TiB 388 MiB 34 GiB 37 TiB 23.61 I'm getting there. Granted, at 3+2 erasure coding that's only a bit over 30TiB usable space. --=20 Rich