From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id D2C83158020 for ; Thu, 8 Dec 2022 13:52:57 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id D74ADE07E0; Thu, 8 Dec 2022 13:52:52 +0000 (UTC) Received: from mail-yb1-f170.google.com (mail-yb1-f170.google.com [209.85.219.170]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 9FF0CE0794 for ; Thu, 8 Dec 2022 13:52:52 +0000 (UTC) Received: by mail-yb1-f170.google.com with SMTP id 7so1672244ybp.13 for ; Thu, 08 Dec 2022 05:52:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5NINUrGOVyHYNiWYAKSuyL+SpOrljqJF8ti5mBi+Gxg=; b=3RvitUaN3NTBfIxK8knUJmkOvXIFzD8HROfqk+jjiz4C87tq5b5QvIXnIYnEUV/XpH 0boBO8DskcIqjpPlnW/YzmqMKHZ3bZP6E6aQYO19/MG0IrH7OJeDYTLhLa2mRhxg8uF9 SA6acRgw7YPe5NZb8XX4+G3MhNy8YoNHDX8cd1X5m7o7jaTZjuVc64onGiLGR81IqpJD T+O2vX2y12PYWom43PC1GtbuqveVtpQe9UwzRZmsk9H8q7E6btEyJtlFSKPs18cxdbHz CAq8kcf8ly3ZgOWgR3YEWiMn6uS2a9FdgzNfrxhZNgAFc0Q2OCKucrz/aInNfw6OyjJs M1RQ== X-Gm-Message-State: ANoB5pnWq8wswmxdszOsDhvU+B7rlVMgmJsLa8ahXyf2zO+hEqNWX04m T5afYE3YuAUQjWhGSCaoy5cU9u2Ttu3v6ruioUAjpdm/rzo= X-Google-Smtp-Source: AA0mqf74u8WXn85dQVZaTJALcTLLPD2p2EtSqgJKbzYSTFumA9YSF7SduKW929j8SWoPWGFENMx+Pi89i2wk34OEAH8= X-Received: by 2002:a25:69d1:0:b0:6f2:883a:7d1a with SMTP id e200-20020a2569d1000000b006f2883a7d1amr62946127ybc.131.1670507571584; Thu, 08 Dec 2022 05:52:51 -0800 (PST) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 References: <9407e524-2226-6ba9-dd7f-bac635d083e3@gmail.com> In-Reply-To: <9407e524-2226-6ba9-dd7f-bac635d083e3@gmail.com> From: Rich Freeman Date: Thu, 8 Dec 2022 08:52:45 -0500 Message-ID: Subject: Re: [gentoo-user] NAS and replacing with larger drives To: gentoo-user@lists.gentoo.org Content-Type: text/plain; charset="UTF-8" X-Archives-Salt: 0e851ef1-5c08-4b93-9404-93f1c1f04f25 X-Archives-Hash: f6bcb8148ec814fd0b56f7fe2bdb17b2 On Thu, Dec 8, 2022 at 7:37 AM Dale wrote: > > Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as > another option. They come as parts, cases too, but the newer and faster > models of Raspberry Pi 4 with more ram seem to work pretty well. For this sort of application the key improvement of the Pi4 over its predecessors is IO. The Pi4 has USB3 and gigabit ethernet, and they are independent, so you get the full bandwidth of both (in theory). That is a massive step up over USB2 and 100Mbps ethernet that consumes the USB2 bandwidth. I can't really speak to the commercial solutions as I haven't used them. Main concern there is just the limited capacity, lack of expandability, and so on. Some are no doubt better than others in those regards. As far as DIY goes, you can definitely do all of that with a Pi4. Don't expect it to perform as well as sticking it on a decent amd64 motherboard, but for backup and saturating the throughput of 1 hard drive at a time it can probably mostly make do. Encryption can be accomplished either with cryptsetup or a filesystem that has native encryption like ZFS. I've done both on Pi4s for storage. I will warn you that zfs encryption is not hardware-optimized on ARM, so that will not perform very well - it will be completely functional, but you will get CPU-bound. Linux-native encryption (ie cryptsetup/LUKS) will use hardware capabilities on the Pi4, assuming you're using something it supports (I think I'm using AES which performs adequately). For the Pi4 you would need to use USB storage, but for hard drives IMO this is perfectly acceptable, especially on a Pi. The gigabit ethernet and internal IO of the Pi is only going to max out one hard drive no matter how you connect it, so the USB3 interface will not be a bottleneck. On ARM SBCs that have PCIe you don't really get any better performance with an HBA and SATA/SCSI simply because the board IO is already pretty limited. USB3 is actually pretty fast for spinning disks, but depending on the number of hosts/etc it could become a bottleneck on a decent motherboard with a large number of drives. If you're talking about an amd64 with a 10GbE NIC and a decent HBA with sufficient PCIe lanes for both then obviously that is going to saturate more spinning disks. For NVMe you absolutely need to go that route (probably need to consider server-class hardware too). I use USB3 hard drives on Pis for my bulk storage because I care about capacity far more than performance, and with a distributed filesystem the performance is still good enough for what I'm doing. If I needed block storage for containers/VMs/whatever then use a different solution, but that gets expensive fast. Oh, one other thing. One of your issues is that you're using a backup solution that just dumps everything into a single file/directory and requires all the backup storage to be mounted at the same time in a single filesystem. There are solutions that do not have this requirement - particularly ones that are adaptable to tape. Unfortunately the best FOSS option I've found for this on linux is bacula and that is a serious PITA to use. If anybody has a better one I'm all ears (the requirement is to be able to store a backup across multiple hard drives, and this can't involve first storing it all in one place and then splitting it up later, or having more than one storage drive attached at the same time - basically I want to treat hard drives like tapes). If you're storing a LOT of backups then LTO is another option. Every time I do the math on that option it never makes sense unless you're backing up a LOT of data. If you got to a point where your backups consumed 10+ max-capacity hard drives it might start to make sense. Those USB3 hard drives on sale for $15/TB though are just really hard to beat when the tapes aren't all that much cheaper and the drives cost $1k. -- Rich