From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 842671382C5 for ; Wed, 31 Mar 2021 11:21:54 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id D183AE0849; Wed, 31 Mar 2021 11:21:47 +0000 (UTC) Received: from mail-gw.thundermail.uk (mail-gw.thundermail.uk [149.255.60.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 78EC3E0839 for ; Wed, 31 Mar 2021 11:21:47 +0000 (UTC) Received: from mailgw01.thundermail.uk (mail-gw.thundermail.uk [149.255.60.66]) by mail-gw.thundermail.uk (Postfix) with ESMTPS id 8ED896003D51 for ; Wed, 31 Mar 2021 12:21:45 +0100 (BST) X-ASG-Debug-ID: 1617189704-05541318051c1930001-LfjuLa Received: from cloud220.unlimitedwebhosting.co.uk (cloud220.unlimitedwebhosting.co.uk [149.255.60.183]) by mailgw01.thundermail.uk with ESMTP id SVelMgu4ITxhUFnV (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 31 Mar 2021 12:21:44 +0100 (BST) X-Barracuda-Envelope-From: confabulate@kintzios.com X-Barracuda-Effective-Source-IP: cloud220.unlimitedwebhosting.co.uk[149.255.60.183] X-Barracuda-Apparent-Source-IP: 149.255.60.183 Received: from lenovo.localdomain (230.3.169.217.in-addr.arpa [217.169.3.230]) by cloud220.unlimitedwebhosting.co.uk (Postfix) with ESMTPSA id 42B2DC856ED for ; Wed, 31 Mar 2021 12:21:43 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kintzios.com; s=default; t=1617189703; bh=qtercTwU0O3UH+OpHMVSl94Aec5+07hayivODeAVNso=; h=From:To:Subject; b=GWj1KIQIoz3WADxypqNBXPvR0DDMxe1wsaRT1w11X+QTfoNhjYF7iw94XQjP7HT9Z k+Wjqv+afFgoj3BJlEMOeTs2KHISquI2sqSphBSxv/2K+hkbPuIC6oU0mkrSkn2gd9 85q0hDAkoCemQW6jyURzLYX+xD/QBVFmDt2/Vp0U= From: Michael To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel? Date: Wed, 31 Mar 2021 12:21:27 +0100 X-ASG-Orig-Subj: Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel? Message-ID: <808599453.0ifERbkFSE@lenovo.localdomain> In-Reply-To: <24675.23516.469155.831394@tux.local> References: <24510.38475.914653.374734@tux.speedport.ip> <24523.52343.49602.311308@tux.speedport.ip> <24675.23516.469155.831394@tux.local> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart5154895.Sb9uPGUboI"; micalg="pgp-sha256"; protocol="application/pgp-signature" X-PPP-Message-ID: <20210331112143.2509962.91566@cloud220.unlimitedwebhosting.co.uk> X-PPP-Vhost: kintzios.com X-Barracuda-Connect: cloud220.unlimitedwebhosting.co.uk[149.255.60.183] X-Barracuda-Start-Time: 1617189704 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://149.255.60.66:443/cgi-mod/mark.cgi X-ASG-Orig-Subj: Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel? X-Virus-Scanned: by bsmtpd at thundermail.uk X-Barracuda-Scan-Msg-Size: 6034 X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=1.9 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.88914 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Archives-Salt: ced5e57d-29c3-4c47-a1af-5ebf1a571d61 X-Archives-Hash: 63e98948a208335c2ec9216441c39eda --nextPart5154895.Sb9uPGUboI Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="UTF-8"; protected-headers="v1" From: Michael To: gentoo-user@lists.gentoo.org Reply-To: confabulate@kintzios.com Subject: Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel? Date: Wed, 31 Mar 2021 12:21:27 +0100 Message-ID: <808599453.0ifERbkFSE@lenovo.localdomain> In-Reply-To: <24675.23516.469155.831394@tux.local> References: <24510.38475.914653.374734@tux.speedport.ip> <24523.52343.49602.311308@tux.speedport.ip> <24675.23516.469155.831394@tux.local> On Tuesday, 30 March 2021 18:11:56 BST Dr Rainer Woitok wrote: > On Saturday, 2020-12-05 19:07:51 +0100, I myself wrote: > > ("> >" refers to Michael ) > > > Michael, > > > > On Friday, 2020-11-27 19:07:17 +0000, you wrote: > > > ... > > > A 4k block size is recommended for ntfs-3g which is the default sector > > > created by fdisk and friends on Linux these days. This will align your > > > partition optimally. In addition, mkfs.ntfs will use 4096 bytes as the > > > default cluster size, so you should be good in that respect. > > > > > > Another setting you may want to try is mounting the USB with > > > 'big_writes' - > > > check the man page. This should help particularly with large files, > > > which > > > will use larger blocks up to 128KB when copying data to the NTFS. > > > > Both, the VeraCrypt command line (--fs-options=big_writes) and the Vera- > > Crypt GUI (under "Settings --> Preferences") allow setting this mount > > option. But > > > > $ mount | grep veracrypt > > > > never shows it, initially causing me to erroneously believe it wasn't > > set and to try finding on the web another way of setting it. By pure > > chance I finally found out that > > > > $ ps -ef | grep veracrypt > > > > lists a "/usr/sbin/mount.ntfs" task which shows the options really in > > effect. However, I haven't yet had the time to test the effect of this > > option when writing plenty of really big files. I will report on that > > later. > > Well, it's been quite a while, due to my being almost permanently con- > fronted with more pressing tasks ... :-( > > To sum up my experience with my new 128 GB Philips USB 3.0 sticks: while > the Philips sticks are significantly faster for reading operations than > my old 64 GB Verbatim ones (probably USB 2.0), writing operations to the > Philips sticks are unbearably slow, regardless of whether I created a > normal unencrypted NTFS filesystem on them or an encrypted NTFS filesys- > tem using VeraCrypt. Writing to the USB stick while at the same time > reading from it in a different terminal window caused commands like "cd" > or "ls" to simply stall. Thus while running > > $ cp --preserve=timestamps -ru $source_dir . > > in one terminal window, I ran > > $ while true > > > do n=$(ps -ef|g 'cp --preserve'|g -v grep) > > > > if [[ "$n" = "${o-}" ]] > > then sleep 10 > > else o="$n" > > > > echo "$n" > > > > fi > > > > done > > in another, to get the wall clock times when copying a new file began. > That way I found that copying a 30 MB file took about 40 minutes. OK, unless you made a typo and the "minutes" were meant to say seconds, this is ridiculously slow. You could run some tests to see what is causing the delay. The veracrypt algos & cipher iterations, the fuse based ntfs-3g, or the USB stick's controller. However if, as I understand it, all other variables are the same and the only change was to replace your Verbatim 64G USB 2.0 sticks with Philips 128G USB 3.0 sticks, then the slow writes point to the Philips devices being the culprit. Some years ago I tested some USB 2.0 sticks of various sizes, from 256M up to 32G and recall the smaller the USB stick the faster the write performance, so differences in writing speed are normal. The writing speed you're describing however is a clear indication of something being wrong. > So what are my options? > > - Stay away from Philips USB 3.0 sticks? > > - Stay away from Philips USB sticks in general? Without knowing the internals, a brand may offer only an unwarranted assumption of performance. We saw Western Digital disks being sold as CMR, while having SMR internals. A brand could switch OEM suppliers, or components, making benchmarking unreliable. > - Stay away from USB 3.0 sticks in general? USB 3.0 is faster and USB 3.2 when available will be even faster. So use whatever the USB ports on your PC offer. > - Stay away from Filesystem in User Space using a non-stable 5.10 or > 5.11 kernel (currently I'm using stable 5.4.97)? > > - Stay away from Gentoo? > > - Stay away from Linux in general and go back to OTOS (aka the Only > True Operating System aka Windoze)? > > - ...? In-kernel fs drivers are measurably faster than fuse based fs for well understood reasons. However, if needs must and the fs you require is not available on Linux, then some compromise will be required. > Any ideas and comments welcome ... > > Sincerely, > Rainer You may want to run some tests on the sticks you have, if only to bottom out what their performance is on different PCs and USB ports: dd if=/dev/zero of=/run/media///TESTFILE bs=512 count=600000 oflag=direct conv=notrunc,fsync status=progress Use a large enough file to make sure the USB controller cache gets saturated. You could use a ramdisk/tmpfs as an input file. If you write directly to the device as Dale suggested it will wipe data, so keep a backup of anything you need first. You can experiment with different filesystems and in the first opportunity with a different make of USB 3.0 stick. You'd soon be able to determine how good the real world performance can get and if the Philips or something else is causing the problem you've experienced. A note about UDF: it works and it is versatile - but ensuring interoperability between different OSs can be tricky. Check some suggestions here: https://askubuntu.com/questions/27936/can-and-should-udf-be-used-as-a-hard-drive-format and here: https://superuser.com/questions/39942/using-udf-on-a-usb-flash-drive Ultimately such tests are an attempt to eliminate methodically all other factors, until you isolate the cause of the problem you are experiencing. --nextPart5154895.Sb9uPGUboI Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEXqhvaVh2ERicA8Ceseqq9sKVZxkFAmBkWzcACgkQseqq9sKV ZxlyXhAA5bk/zLyItMFH+S4ZU4+UVdomMxxzlsnTsmre42jPHZhe9NfUFT0FjsEw ZCTZNp+IhmRgoMy6B0mvshdVPXjPidrH7QQYv1wN0nStjcnFC0A+ytC/sIly7PBQ o13h8REzo/fBYKffFLPx121TBkgGUT7r1RYJsyG15rGiJdUGKPyKnfK6nwJv6Sf1 SHWFJ8i9loRtKhpC8D0+PjTrDkenDrNj5ojwyf8iKIBrwSGIEAN8eJy08HeYlMrC 1rBC9eBin4Iu8Y1z0ih/gDrItNcZY7q5+N+0Htf5t/TLjU5YSp8OjYO1sGye++WL MoIKC6fc+q6jqsVQHyPxcI7n188xVo1Dcu5/M52xmAsTaVNAFz6cmst43feBKWPi 3fe4IscJa2BbabulU1w6cdQ0GVgeTAgds8myy+8qFZPdLQ2HEgI1UfTpng2QgaYN JCaLkfj/u/sBJxRJ3aFdnRiT7C1+EyjZ2tM/mU0VImkZ9Flwdj3FgACb6VrrPclU x2YHi8P5o5XoMIYMSsElToXAlAe8LsMPQ8lU/fVv8M3l8/kTb6pfEU4WQgDzuYrl AVFWecrqBv1YZBKqStRTUzxfCC5V5YpgqdtQVGoyZkk9CLE+dzHM9a1dCJFzRAyz 0y5J2XczhaaAU0KOE4Du+RttU/3kWAh5xGPfFvTabTuLVIjZh+M= =otvP -----END PGP SIGNATURE----- --nextPart5154895.Sb9uPGUboI--