Re: [gentoo-user] Copying data efficiently

[meino.cramer@gmx.de]

Daniel Troeder <daniel@admin-box.com> [14-05-17 16:36]:
> Am 17.05.2014 11:58, schrieb Neil Bothwick:
> > On Sat, 17 May 2014 09:59:08 +0200, meino.cramer@gmx.de wrote:
> > 
> >>> 3. Or you could use a sequential copy:
> >>>
> >>>   cp -a /home /dev/sdb1/ && cp -a /home /dev/sdc1
> > 
> >> 3.) The files I want to copy are in the size of some GB each. So the
> >>     cache isnt big enough to hold ALL files for the second part.
> > 
> > Run the two copies simultaneously, start the first, switch to another
> > tab, start the second. That way the data for the second copy is always
> > the most recently cached.
> > 
> > However, I expect the speed limit here may be the USB bus unless you are
> > using USB 3.0 drives on different buses.
> I was thinking about "how to make sure cache is used", and that two
> simultanious cp won't work, because the progress for the two cp will
> quickly diverge. But then I realized: there is no need to think about
> the read cache - the limiting factor is always the writing side,
> especially with USB! So IMO it doesn't matter at all how you do it!
> 
> I guess two simultaneous cp will be the same as two sequential cp,
> except if you have two separate USB-buses. Usually you have just one
> externally connectible, use "lsusb -t" to check.
> 
> If you have less that 2 times the size of your files, IMO simultaneous
> cp will be worse, because Linux (don't know if USB-subsystem or cp)
> creates big buffers when cp'ing (check with "free -m"), and you'll
> probably get into memory trouble.
> 
> 
> Greetings,
> Daniel
> 
> PS: Quickest way is always to open USB-case and plug SATA cable from
> motherboard into drive. With >80GB it's always worth the trouble.
> 
> -- 
> Get my PGP key at:
> *
> http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x837FB8B5BB9D4887
> * $ gpg --recv-keys --keyserver keyserver.ubuntu.com 0xBB9D4887
> 

Hi,

thank you /.*/  for /.*/ replies ! :)

...one aspect is missing: The load (that is the I/O on the 
source hd).
If this hd is busy spitting out the data twice, it cannot
serve outhe jobs twice as long...

Hmmmm....

Best regards,
mcc

PS:
On the source system
#> lsusb -t 
/:  Bus 09.Port 1: Dev 1, Class=root_hub, Driver=ohci-pci/4p, 12M
/:  Bus 08.Port 1: Dev 1, Class=root_hub, Driver=ohci-pci/2p, 12M
/:  Bus 07.Port 1: Dev 1, Class=root_hub, Driver=ohci-pci/5p, 12M
    |__ Port 2: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
    |__ Port 2: Dev 2, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M
/:  Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 5000M
/:  Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=ohci-pci/5p, 12M
    |__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
    |__ Port 3: Dev 3, If 0, Class=Audio, Driver=snd-usb-audio, 12M
    |__ Port 3: Dev 3, If 1, Class=Audio, Driver=snd-usb-audio, 12M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/4p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/5p, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/5p, 480M
    |__ Port 4: Dev 4, If 0, Class=Hub, Driver=hub/6p, 480M