Re: [gentoo-user] NAS and replacing with larger drives

[Michael <confabulate@kintzios.com>]

[-- Attachment #1: Type: text/plain, Size: 6585 bytes --]

On Thursday, 8 December 2022 20:44:56 GMT Frank Steinmetzger wrote:
> Am Thu, Dec 08, 2022 at 06:36:14PM +0000 schrieb Wols Lists:
> > >  > I've pretty much reached a limit on my backups.  I'm up to a 16TB
> > >  > hard
> > >  > drive for one and even that won't last long.  Larger drives are much
> > >  > more costly.  A must have NAS is quickly approaching.  I've been
> > >  > searching around and find some things confusing.  I'm hoping someone
> > >  > can
> > >  > clear up that confusion.  I'm also debating what path to travel down.
> > >  > I'd also like to keep costs down as well.  That said, I don't mind
> > >  > paying a little more for one that would offer a much better option.
> > >  > 
> > >  > Path one, buy a NAS, possibly used, that has no drives.  If possible,
> > >  > I
> > >  > may even replace the OS that comes on it or upgrade if I can.  I'm
> > >  > not
> > >  > looking for fancy, or even RAID.  Just looking for a two bay NAS that
> > >  > will work.  First, what is a DAS?  Is that totally different than a
> > >  > NAS?  From what I've found, a DAS is not what I'm looking for since I
> > >  > want a ethernet connection and the ability to control things over the
> > >  > network.  It seems DAS lacks that feature but not real sure.  I'm not
> > >  > sure I can upgrade the software/OS on a DAS either.
> > >  > […]
> > > 
> > > DAS is direct-attached-storage. I don't think you want that.
> > 
> > Depends. If it fits in the safe, and can be connected using one of these
> > eSATA thingy connectors, it might be a very good choice.
> > 
> > […]
> > 
> > I get the impression Dale isn't actually PLANNING his disk storage. It's
> > just a case of "help I'm downloading all this stuff where do I put it!!!"
> 
> Haha, thanks for the laugh.

Actually this had me thinking what is the need to back up the ... Internet?  
If all this never-ending and recently accelerated download activity by Dale 
will continue and most of these video/audio files are available on some 
streaming server on the Internet, WHY do they need to be backed up locally?

I appreciate some of these video files may be rare finds, or there may be a 
risk some of these may be taken off the interwebs sooner or later.  This 
should leave a rather small subset of all downloads, which may merit a local 
backup, just in case.  I'd thought the availability of higher fiber download 
speeds negates the need for local backups, of readily downloadable media.

Of course, with personal and private data, plus configuration files, the 
backup need is clearer and the strategy simpler.

Perhaps the whole backup strategy for files downloaded from the Internet, Vs 
personal files, needs some critical (re)thinking.


> > Get yourself a basic 4-way DAS/JBOD setup, PLAN where you're putting all
> > this stuff, and plug in and remove drives as required. You don't need all
> > these huge drives if you think about what you're going to do with it all.
> 
> That’s actually a good idea. Either use a hot swap frame for an internal 5¼″
> PC bay, a desktop dock for bare drives or a multi-bay enclosure. The market
> is big, you have lots of choices. USB (with or without integrated hub),
> eSATA, one or two bays, etc: https://skinflint.co.uk/?cat=hddocks
> 
> Advantages:
> - no separate system to maintain just for storage: save $$$, time and power
> - very flexible: no chassis limitation on number of disks
> - no bulky external enclosures, each using a different power brick and cable
> - minimum volume to put into a safe (just get or make a bulk storage case)
> 
> Disadvantages:
> - not as “fancy” as a NAS
> - possibly not all disks can be used at the same time
> - physical handling of naked disks takes more care
> - LVM is not practical, so use each disk separately
> - you gotta remember which files are where¹
> - SATA connectors aren’t made for very many insertion cycles (I think the
>   spec says 50?), which doesn’t mean they endure much more, but still …
> 
> > (And while it takes time and hammers the system, I regularly record off
> > the
> > TV getting a 2GB .ts file, convert it to mp4 - same resolution - and
> > reduce
> > the size by an order of magnitude - maybe more.
> 
> Well, ts uses mpeg2 encoding, just like old video DVDs, which is very
> inefficient when compared with modern h264/h265. Modern digital TV broadcast
> uses h264 by now.

Depending on the PVR make/model I've seen 1080p resolution recordings with 
.m2ts and .ts file extensions, while the codecs inside them are the same.  
Here's the ffprobe output of a .ts recording containing a h264 video I 
captured recently off terrestrial TV transmission:

Input #0, mpegts, from '20221209 0147 - BBC ONE HD - Weather for the Week 
Ahead.ts':
  Duration: 00:13:01.44, start: 48999.919856, bitrate: 3744 kb/s
  Program 17540 
  Stream #0:0[0x19c9]: Video: h264 (High) ([27][0][0][0] / 0x001B), 
yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 
90k tbn, 50 tbc
  Stream #0:1[0x19ca](eng): Audio: aac_latm (LC) ([17][0][0][0] / 0x0011), 
48000 Hz, 5.1, fltp
  Stream #0:2[0x19ce](eng): Unknown: none ([17][0][0][0] / 0x0011) (visual 
impaired) (descriptions) (dependent)
  Stream #0:3[0x19cd](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)

The h264 codec means converting the .ts file to mp4 will not change much at 
all the size of the file.  If anything the mp4 file could increase in size if 
the audio stream and subtitles need to be transcoded.


> Incidentally, I got myself a new HDD today: an external 2.5″ WD Passport
> Ultra 5 TB with USB-C 3.0. Just because I like portable storage and also
> because I need temporary space if I want to convert my NAS RAID-Z2 to Z1.
> 
> 
> ¹ I do have several external USB disks, plus the big NAS. All of which don’t
> run very often. And I don’t want to turn them on just to look for a certain
> file. That’s why I have another little script. ;-) It uses the `tree`
> command to save the complete content listing of a directory into a text
> file and names the file automatically by the name of the directory it
> crawls. So if I want to find a file, I just need to grep through my text
> files.

Backup scripts utilising rsync, tar, etc. can output a log file which contains 
(some) details of all the backed up files.  Nothing as sophisticated as 
Frank's script, but it allows for a quick search against the name of the file 
or directory, before extraction.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]