From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 365061382C5 for ; Sun, 30 May 2021 00:42:45 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 1C042E0823; Sun, 30 May 2021 00:42:41 +0000 (UTC) Received: from mail143c7.megamailservers.com (mail119c7-2520.megamailservers.com [69.49.98.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id AC2E2E07FE for ; Sun, 30 May 2021 00:42:40 +0000 (UTC) X-Authenticated-User: admin@sys-concept.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=megamailservers.com; s=mailtor; t=1622335360; bh=gSr+GjElkJYWzMlgYPMGfmBg0pCJ7RfD0lE/irBUSSM=; h=To:References:From:Subject:Date:In-Reply-To:From; b=KZeahqahnFlNjs86N3wpIDuFdvG9lvFwZFKm0UCbTahe4UF6dZymUE9ZG7LCtrWfn 63MoySyce7OjG7tF1hlVBTlCteZcQqAFp7BNdSx5sqgKHiRlV+mpEQdSQxDsA3wHHO pvHDI5SA5G78LcRM6UFO2TgscOEiNvvqNBxt4J7g= Feedback-ID:thelma@sys-conc X-VIP: 69.49.109.100 Received: from [10.0.0.109] 184.69.242.18":"36587 ([184.69.242.18]) (authenticated bits=0) by mail143c7.megamailservers.com (8.14.9/8.13.1) with ESMTP id 14U0gcWn016901 for ; Sat, 29 May 2021 20:42:39 -0400 To: gentoo-user@lists.gentoo.org References: <1eee5cf4-9234-07cf-4b83-7de431483d51@sys-concept.com> <8a5af6e4-0544-df31-98aa-58dbddfa1608@mail.meme.technology> From: thelma@sys-concept.com Subject: Re: [gentoo-user] is "scp" reliable? Message-ID: <8e8edccb-5266-0d8f-4a67-0e77859c34a7@sys-concept.com> Date: Sat, 29 May 2021 18:42:35 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 In-Reply-To: <8a5af6e4-0544-df31-98aa-58dbddfa1608@mail.meme.technology> Content-Type: text/plain; charset=utf-8 Content-Language: en-CA Content-Transfer-Encoding: 8bit X-CTCH-RefID: str=0001.0A742F21.60B2DF80.0004:SCFSTAT54230205,ss=1,re=-4.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0 X-CTCH-VOD: Unknown X-CTCH-Spam: Unknown X-CTCH-Score: -4.000 X-CTCH-Rules: X-CTCH-Flags: 0 X-CTCH-ScoreCust: 0.000 X-CSC: 0 X-CHA: v=2.3 cv=FO9lONgs c=1 sm=1 tr=0 a=u3gW5Uk2xAGXuEgp5FffjQ==:117 a=u3gW5Uk2xAGXuEgp5FffjQ==:17 a=IkcTkHD0fZMA:10 a=qMvgliNbSE_S7-RKj4wA:9 a=QEXdDO2ut3YA:10 X-VADE-SPAMSTATE: clean X-VADE-SPAMSCORE: 0 X-VADE-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduledrvdeltddgfeeiucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecujffquffvqffrkfetpdfqfgfvpdevtdejnecuuegrihhlohhuthemuceftddunecunecujfgurhepvfhfhffukffffgggjggtgfesthekredttdefjeenucfhrhhomhepthhhvghlmhgrsehshihsqdgtohhntggvphhtrdgtohhmnecuggftrfgrthhtvghrnhepvedtgeefhfeghfdvueffudffudefueeutdfhgfevieelkeehteevvedufefhffejnecukfhppedukeegrdeiledrvdegvddrudeknecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepudekgedrieelrddvgedvrddukedphhgvlhhopegluddtrddtrddtrddutdelngdpmhgrihhlfhhrohhmpehthhgvlhhmrgesshihshdqtghonhgtvghpthdrtghomhdprhgtphhtthhopehgvghnthhoohdquhhsvghrsehlihhsthhsrdhgvghnthhoohdrohhrgh X-Origin-Country: CA X-Archives-Salt: bcb9b7b1-e9af-4ee2-9686-a3926748b0d5 X-Archives-Hash: d839995f60ca45ce65e920d793393ba6 >> Another mystery. >> I copied the file to USB 1TB sandisk. >> md5sum check OK same as my computer >> >> >> md5sum /run/media/joseph/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova >> 6f3348f1fb915af9c45806d947558a37 /run/media/joseph/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova >> >> I mount the same USB 1TB sandisk on another computer and running md5sum on same file gives me different number, why??? >> >> md5sum /run/media/fd/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova >> c478cb48e2f7961cb0e3eb452df6e642 /run/media/fd/SSD-1TB/business/backup/VDI/windows-7_pro_May-23-21.ova >> > Did you sync and unmount the partition before ejecting the drive from > the first computer? With a file this large being copied, it is likely > that a large amount of data remains buffered/cached and will not be > fully written to the flash memory even after the copy command completes. > > On the first machine, you would still see the correct md5sum because the > kernel abstracts this fact away from you. But if you rip out the drive > and take it somewhere else without flushing those caches, you're going > to get an incomplete file. > > Check if the file on the drive still md5sums the same if you plug it > back into the first machine. Check what size it is, and whether there > are a lot of 0s at the end indicating an unfinished write. > > cal Yes, I unmounted the USB device every time. And yes, I plug the USB device back to original machine and md5sum is correct, same as the original. I copied the large file over network to another box and md5sum of: windows-7_pro_May-23-21.ova is correct same as on the original box. I run this: "rsync -avh [source] [destination] && rsync -avhc [source] [destination]" above code rsync files folder on first run and if complete without issue, will run rsync again immediately while performing same file name comparison by using hash of entire file. This i what I got: rsync -avh windows-7_pro_May-29-21.ova fd@10.0.0.138:/home/fd/business/VDI/ && rsync -avhc windows-7_pro_May-29-21.ova fd@10.0.0.138:/home/fd/business/VDI/ sending incremental file list windows-7_pro_May-29-21.ova sent 30.29G bytes received 35 bytes 115.81M bytes/sec total size is 30.28G speedup is 1.00 sending incremental file list windows-7_pro_May-29-21.ova WARNING: windows-7_pro_May-29-21.ova failed verification -- update discarded (will try again). windows-7_pro_May-29-21.ova ERROR: windows-7_pro_May-29-21.ova failed verification -- update discarded. sent 33.44M bytes received 6.47M bytes 123.38K bytes/sec total size is 30.28G speedup is 758.62 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]