From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 5B6BD158020 for ; Fri, 9 Dec 2022 13:14:19 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 62DCDE089A; Fri, 9 Dec 2022 13:14:10 +0000 (UTC) Received: from mail-gw.thundermail.uk (mail-gw.thundermail.uk [149.255.60.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id DE298E086E for ; Fri, 9 Dec 2022 13:14:09 +0000 (UTC) Received: from mailgw01.thundermail.uk (uwhmailrelay01.unlimited.uk.net [149.255.60.67]) by mail-gw.thundermail.uk (Postfix) with ESMTPS id 309706047FD8 for ; Fri, 9 Dec 2022 13:14:08 +0000 (GMT) X-ASG-Debug-ID: 1670591647-0554133dd323bbf10001-LfjuLa Received: from mailclean01.thundermail.uk (mailclean01.thundermail.uk [149.255.60.66]) by mailgw01.thundermail.uk with ESMTP id 13vkjgsV7iDAmGOP (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Fri, 09 Dec 2022 13:14:07 +0000 (GMT) X-Barracuda-Envelope-From: confabulate@kintzios.com X-Barracuda-Effective-Source-IP: mailclean01.thundermail.uk[149.255.60.66] X-Barracuda-Apparent-Source-IP: 149.255.60.66 Received: from cloud220.unlimitedwebhosting.co.uk (cloud220.unlimitedwebhosting.co.uk [149.255.60.183]) by mailclean01.thundermail.uk (Postfix) with ESMTPS id 9AAB9417F8 for ; Fri, 9 Dec 2022 13:14:00 +0000 (GMT) Received: from lenovo.localdomain (230.3.169.217.in-addr.arpa [217.169.3.230]) by cloud220.unlimitedwebhosting.co.uk (Postfix) with ESMTPSA id B0696C0E52B for ; Fri, 9 Dec 2022 13:14:00 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kintzios.com; s=default; t=1670591641; bh=mw+kaBYDk6X2XFxgeQeTDgsf2paL9AW3tRzI+KMVrhw=; h=From:To:Subject; b=YQUkmChMkuEZ+Zsolz5sNvPi2fENCNv5YOfyvugtJv1SwHZ1H7xSfmvpUsfe1bqYa c3SuyXlKxFnnnkafDwYeF1J7cF34JSI8luWK4SEOkk+/+GvoQ/jxmNny+lmGCxABRm kvGIFH8noHvpYCt3Xcoq/TG4Cw8o3fIpAwXp26JY= From: Michael To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] NAS and replacing with larger drives Date: Fri, 09 Dec 2022 13:13:50 +0000 X-ASG-Orig-Subj: Re: [gentoo-user] NAS and replacing with larger drives Message-ID: <2398014.jE0xQCEvom@lenovo.localdomain> In-Reply-To: References: <9407e524-2226-6ba9-dd7f-bac635d083e3@gmail.com> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2328110.NG923GbCHz"; micalg="pgp-sha256"; protocol="application/pgp-signature" X-PPP-Message-ID: <167059164103.3991897.12907177830115646580@cloud220.unlimitedwebhosting.co.uk> X-PPP-Vhost: kintzios.com X-Spam-Status: No, score=-0.2 required=11.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,SPF_PASS autolearn=disabled version=3.4.2 X-Spam-Report: * -0.0 SPF_PASS SPF: sender matches SPF record * -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature * -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from * author's domain * -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from * envelope-from domain * 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily * valid X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mailclean01.unlimited.uk.net X-Barracuda-Connect: mailclean01.thundermail.uk[149.255.60.66] X-Barracuda-Start-Time: 1670591647 X-Barracuda-Encrypted: TLS_AES_256_GCM_SHA384 X-Barracuda-URL: https://149.255.60.67:443/cgi-mod/mark.cgi X-ASG-Orig-Subj: Re: [gentoo-user] NAS and replacing with larger drives X-Virus-Scanned: by bsmtpd at thundermail.uk X-Barracuda-Scan-Msg-Size: 6591 X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.102714 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Archives-Salt: 4a2956e2-e2ec-4bf1-abf0-46bd5da02ba2 X-Archives-Hash: ac19e7e4767a85c0334a4c6506df26f4 --nextPart2328110.NG923GbCHz Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8"; protected-headers="v1" From: Michael To: gentoo-user@lists.gentoo.org Reply-To: confabulate@kintzios.com Subject: Re: [gentoo-user] NAS and replacing with larger drives Date: Fri, 09 Dec 2022 13:13:50 +0000 Message-ID: <2398014.jE0xQCEvom@lenovo.localdomain> In-Reply-To: MIME-Version: 1.0 On Thursday, 8 December 2022 20:44:56 GMT Frank Steinmetzger wrote: > Am Thu, Dec 08, 2022 at 06:36:14PM +0000 schrieb Wols Lists: > > > > I've pretty much reached a limit on my backups. I'm up to a 16TB > > > > hard > > > > drive for one and even that won't last long. Larger drives are mu= ch > > > > more costly. A must have NAS is quickly approaching. I've been > > > > searching around and find some things confusing. I'm hoping someo= ne > > > > can > > > > clear up that confusion. I'm also debating what path to travel do= wn. > > > > I'd also like to keep costs down as well. That said, I don't mind > > > > paying a little more for one that would offer a much better option. > > > >=20 > > > > Path one, buy a NAS, possibly used, that has no drives. If possib= le, > > > > I > > > > may even replace the OS that comes on it or upgrade if I can. I'm > > > > not > > > > looking for fancy, or even RAID. Just looking for a two bay NAS t= hat > > > > will work. First, what is a DAS? Is that totally different than a > > > > NAS? From what I've found, a DAS is not what I'm looking for sinc= e I > > > > want a ethernet connection and the ability to control things over = the > > > > network. It seems DAS lacks that feature but not real sure. I'm = not > > > > sure I can upgrade the software/OS on a DAS either. > > > > [=E2=80=A6] > > >=20 > > > DAS is direct-attached-storage. I don't think you want that. > >=20 > > Depends. If it fits in the safe, and can be connected using one of these > > eSATA thingy connectors, it might be a very good choice. > >=20 > > [=E2=80=A6] > >=20 > > I get the impression Dale isn't actually PLANNING his disk storage. It's > > just a case of "help I'm downloading all this stuff where do I put it!!= !" >=20 > Haha, thanks for the laugh. Actually this had me thinking what is the need to back up the ... Internet?= =20 If all this never-ending and recently accelerated download activity by Dale= =20 will continue and most of these video/audio files are available on some=20 streaming server on the Internet, WHY do they need to be backed up locally? I appreciate some of these video files may be rare finds, or there may be a= =20 risk some of these may be taken off the interwebs sooner or later. This=20 should leave a rather small subset of all downloads, which may merit a loca= l=20 backup, just in case. I'd thought the availability of higher fiber downloa= d=20 speeds negates the need for local backups, of readily downloadable media. Of course, with personal and private data, plus configuration files, the=20 backup need is clearer and the strategy simpler. Perhaps the whole backup strategy for files downloaded from the Internet, V= s=20 personal files, needs some critical (re)thinking. > > Get yourself a basic 4-way DAS/JBOD setup, PLAN where you're putting all > > this stuff, and plug in and remove drives as required. You don't need a= ll > > these huge drives if you think about what you're going to do with it al= l. >=20 > That=E2=80=99s actually a good idea. Either use a hot swap frame for an i= nternal 5=C2=BC=E2=80=B3 > PC bay, a desktop dock for bare drives or a multi-bay enclosure. The mark= et > is big, you have lots of choices. USB (with or without integrated hub), > eSATA, one or two bays, etc: https://skinflint.co.uk/?cat=3Dhddocks >=20 > Advantages: > - no separate system to maintain just for storage: save $$$, time and pow= er > - very flexible: no chassis limitation on number of disks > - no bulky external enclosures, each using a different power brick and ca= ble > - minimum volume to put into a safe (just get or make a bulk storage case) >=20 > Disadvantages: > - not as =E2=80=9Cfancy=E2=80=9D as a NAS > - possibly not all disks can be used at the same time > - physical handling of naked disks takes more care > - LVM is not practical, so use each disk separately > - you gotta remember which files are where=C2=B9 > - SATA connectors aren=E2=80=99t made for very many insertion cycles (I t= hink the > spec says 50?), which doesn=E2=80=99t mean they endure much more, but s= till =E2=80=A6 >=20 > > (And while it takes time and hammers the system, I regularly record off > > the > > TV getting a 2GB .ts file, convert it to mp4 - same resolution - and > > reduce > > the size by an order of magnitude - maybe more. >=20 > Well, ts uses mpeg2 encoding, just like old video DVDs, which is very > inefficient when compared with modern h264/h265. Modern digital TV broadc= ast > uses h264 by now. Depending on the PVR make/model I've seen 1080p resolution recordings with= =20 =2Em2ts and .ts file extensions, while the codecs inside them are the same.= =20 Here's the ffprobe output of a .ts recording containing a h264 video I=20 captured recently off terrestrial TV transmission: Input #0, mpegts, from '20221209 0147 - BBC ONE HD - Weather for the Week=20 Ahead.ts': Duration: 00:13:01.44, start: 48999.919856, bitrate: 3744 kb/s Program 17540=20 Stream #0:0[0x19c9]: Video: h264 (High) ([27][0][0][0] / 0x001B),=20 yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 t= br,=20 90k tbn, 50 tbc Stream #0:1[0x19ca](eng): Audio: aac_latm (LC) ([17][0][0][0] / 0x0011),= =20 48000 Hz, 5.1, fltp Stream #0:2[0x19ce](eng): Unknown: none ([17][0][0][0] / 0x0011) (visual= =20 impaired) (descriptions) (dependent) Stream #0:3[0x19cd](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) The h264 codec means converting the .ts file to mp4 will not change much at= =20 all the size of the file. If anything the mp4 file could increase in size = if=20 the audio stream and subtitles need to be transcoded. > Incidentally, I got myself a new HDD today: an external 2.5=E2=80=B3 WD P= assport > Ultra 5 TB with USB-C 3.0. Just because I like portable storage and also > because I need temporary space if I want to convert my NAS RAID-Z2 to Z1. >=20 >=20 > =C2=B9 I do have several external USB disks, plus the big NAS. All of whi= ch don=E2=80=99t > run very often. And I don=E2=80=99t want to turn them on just to look for= a certain > file. That=E2=80=99s why I have another little script. ;-) It uses the `t= ree` > command to save the complete content listing of a directory into a text > file and names the file automatically by the name of the directory it > crawls. So if I want to find a file, I just need to grep through my text > files. Backup scripts utilising rsync, tar, etc. can output a log file which conta= ins=20 (some) details of all the backed up files. Nothing as sophisticated as=20 =46rank's script, but it allows for a quick search against the name of the = file=20 or directory, before extraction. --nextPart2328110.NG923GbCHz Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEXqhvaVh2ERicA8Ceseqq9sKVZxkFAmOTNI4ACgkQseqq9sKV ZxmakhAAjnlF09MOgsYixAL3Q0/vAdHkwFJjY2vmqy+0NDWJQZCSm18i2of6hsyT a0lQsyXODdFTs59gUyjCXxEGpq37wzOBD9osPbmZKpN4pEKXX7mme4g/rCBAsLGK 0ATfGoLAZsKXGm9shOBn8CoV2lmF/+jLulChrznvF1jJSa3sekNgHl9kq0ViSQNf UfvU/xT9pM2XFNE8pm72DFM33F7oF4PrAGwaEkvgs+koLzYuLYtf6UPLWIETC+92 Xh7RkIq5bsGgvwTTrUY9mXXzty8Zdz56xvCAkewE4M2UeEzPuaJgc3YbKuocEAIU GuZz9LghZqPtxwKQU97yoXH2gGaJB2Z9iWVzR39ddK33keCklr182mXsMjQauAq1 n533qfRjWPZc9JX5dplAL47EOIkn7DlIcmlkbQQG1XlSwm7ByCrtjuL0hOrAm6JI hmTDVtcRmdLd60fdSC8qa0iSVRp0fNvSngoMzIKrZ8lHUEZMTgNd3VQe9cY3gp+5 Z1b2zMc+ccdelI0/V20FPI1ik2YQbwGjsnURSQH2No8t2cHoahLV24AGYuucI7r7 Q4wA5ou+LFwR7bPYnfXCxmNy0J5m14CWY2qhn/x/lvm0TipB83kkpc33cSEyMTZQ FFFMm6ooABmtcRnYbeeSWNWfof3tkT6wx7KdGmuqVOxGfwkZiFo= =GdXB -----END PGP SIGNATURE----- --nextPart2328110.NG923GbCHz--