From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 570DE13933E for ; Thu, 1 Jul 2021 13:47:20 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 6BEB8E086D; Thu, 1 Jul 2021 13:47:13 +0000 (UTC) Received: from mout01.posteo.de (mout01.posteo.de [185.67.36.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 946CEE0837 for ; Thu, 1 Jul 2021 13:47:11 +0000 (UTC) Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 066D3240029 for ; Thu, 1 Jul 2021 15:47:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1625147230; bh=N5IfLSNPy5przd94QBvE+PakJ4MXQEhtJbB6GzSuYZI=; h=From:To:Subject:Date:From; b=GfV5hajS2ywCskJNcPbyrWux2vlgZk0Kq+a1E0ywLeYU14BWNjp1VWxZns7MVu2Mc CKEOP5dReYv7x9DGQxXqeDCt4cZe4iNDywK3NsPijnkaUVxKWYuPuL4qBN41tGOm9j 3ZxMEaDYDqDS4UEpSv8hbq6tMJaXqq7hcyKjgfBseAkAd++KOWxVwGBVTqNbO+lYpU rbLUZ2MQea8D5DQ+g41fL8AT82vx0v0Q2K8dsA8brzQtKxSpM6yFiP6O1CxCga/L7q NOtxVccfWipDSI/ihG5Rf/c5KYVHkyuM39XdhM0P7hgW9+4RanW/M/YtrjEW8Ctswd CmRojzwkTEgfQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4GFzzK3qZGz6tmF for ; Thu, 1 Jul 2021 15:47:09 +0200 (CEST) From: Robert David To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] [OT] Using an odd number of drives in ZFS RaidZ Date: Thu, 01 Jul 2021 13:47:08 +0000 Message-ID: <7622209.dfAi7KttbR@robert-notebook> In-Reply-To: References: Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" X-Archives-Salt: 3e77db17-e592-41d2-8f6e-878b0bfcf678 X-Archives-Hash: 9de5a864471dd84819531783aba15d28 Hi Frank, On Tuesday, June 29, 2021 3:56:49 PM CEST Frank Steinmetzger wrote: > Hello fellows >=20 > This is not really a Gentoo question, but at least my NAS (which this mail > is about) is running Gentoo. :) >=20 > There are some people amongst this esteemed group that know their stuff > about storage and servers and things, so I thought I might try my luck he= re. > I=E2=80=99ve already looked on the Webs, but my question is a wee bit spe= cific and > I wasn=E2=80=99t able to find the exact answer (yet). And I=E2=80=99m a b= it hesitant to ask > this newbie-ish question in a ZFS expert forum. ;-) >=20 > Prologue: > Due to how records are distributed across blocks in a parity-based ZFS vd= ev, > it is recommended to use 2^n data disks. Technically, it is perfectly fine > to deviate from it, but for performance reasons (mostly space efficiency) > it is not the recommended way. That=E2=80=99s because the (default) maxim= um record > size of 128 k itself is a power of 2 and thus can be distributed evenly on > all drives. At least that=E2=80=99s my understanding. Is that correct? >=20 > So here=E2=80=99s the question: > If I had three data drives, (c|w)ould I get around that problem by settin= g a > record size that is divisible by 3, like 96 k, or even 3 M? I would not bother with this. 128k is a good default for general usage and even if you got 3 data disks the actual loss is pointless to think about (assuming you got 4k disks). >=20 >=20 >=20 > Here=E2=80=99s the background of my question: > Said NAS is based on a Mini-ITX case which has only four drive slots (whi= ch > is the most common configuration for a case of this formfactor). I started > with two 6 TB drives, running in a mirror configuration. One year later > space was running out and I filled the remaining slots. To maximise > reliability, I went with RaidZ2. >=20 > I reached 80 % usage (which is the recommended maximum for ZFS) and am > now evaluating my options for the coming years. > 1) Reduce use of space by re-encoding. My payload is mainly movies, among > which are 3 TB of DVDs which can be shrunk by at least =E2=85=94 by re= =2Dencoding. > =E2=86=92 this takes time and computing effort, but is a long-term goa= l anyway. I always think about in such cases if I really need such data. In many cases with clear consideration I find out I may remove half of the data without any pain. It is like cleaning my home, there are many things extra and there is missing a space for real valuable things, with disk data it is the same. > 2) Replace all drives with bigger ones. There are three counter arguments: > =E2=80=A2 1000 =E2=82=AC for four 10 TB drives (the biggest size avail= able w/o helium) > =E2=80=A2 they are only available with 7200 rpm (more power, noise and= heat) > =E2=80=A2 I am left with four perfectly fine 6 TB drives > 3) Go for 4+2 RaidZ2. This requires a bigger case (with new PSU due to > different form factor) and a SATA expansion card b/c the Mobo only has > six connectors (I need at least one more for the system drive), costing > 250 =E2=82=AC plus drives. > 4) Convert to RaidZ1. Gain space of one drive at the cost of resilience. I > can live with the latter; the server only runs occasionally and not for > very long at a time. *** This option brings me to my question above, > because it is easy to achieve and costs no =E2=82=AC=E2=82=AC=E2=82=AC. In any of my data arrays I have long time migrated off the RAIDZ to the MIRROR or RAID10. You will find finally that the RAIDZ is slow and not very flexible. Only think you gain is the extra space in constrained array spaces. For RAID10 it is much easier to raise the size, just resilvering to new bigger disks, removing old and expanding. The resilvering speed is magnitude faster. And anyway much easier to recover in cases of failure.=20 If you really need the additional space, consider adding second jbod with another disks. Robert.