[gentoo-user] problem formatting new 256 GB USB stick

[gentoo-user] problem formatting new 256 GB USB stick

[Philip Webb]

Recently, I bought  2  new Kingston Exodia  256 GB  USB sticks
from Canada Computers, the store in Toronto I've used for  25 yr .
With many previous new USB sticks of sizes  <= 128 GB
& which came with a VFat filesystem,
I simply repartitioned them using Fdisk, which created a Linux partition
& then used 'mke2fs' to format them with an Ext2 filesystem.
This time, something has gone wrong :

  root:538 ~> mke2fs /dev/sdb1
  mke2fs 1.47.0 (5-Feb-2023)
  Creating filesystem with 60567296 4k blocks and 15147008 inodes
  Filesystem UUID: 80c2f275-ed6b-4ef5-b785-b53bd225ca9e
  Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912,
    819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000,
    23887872
  Allocating group tables: done                            
  Writing inode tables: done                            
  Writing superblocks and filesystem accounting information:
    mke2fs: Input/output error while writing out and closing file system

I tried repartitioning the stick into  2 x 128 GB  partitions,
in case it was the sheer size which was the problem, but got the same result.
The error occured with both sticks, so it doesn't seem to be bad hardware.
It took  10 h 40 m  to process the  256 GB  part'n on my 2023 desktop machine,
so trying suggestions cb a rather long-drawn-out affair (smile).

Has anyone else encountered this ?  Does anyone have suggestions ?

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

[gentoo-user] Re: problem formatting new 256 GB USB stick

[Nuno Silva]

On 2025-02-15, Philip Webb wrote:

> Recently, I bought  2  new Kingston Exodia  256 GB  USB sticks
> from Canada Computers, the store in Toronto I've used for  25 yr .
> With many previous new USB sticks of sizes  <= 128 GB
> & which came with a VFat filesystem,
> I simply repartitioned them using Fdisk, which created a Linux partition
> & then used 'mke2fs' to format them with an Ext2 filesystem.
> This time, something has gone wrong :
>
>   root:538 ~> mke2fs /dev/sdb1
>   mke2fs 1.47.0 (5-Feb-2023)
>   Creating filesystem with 60567296 4k blocks and 15147008 inodes
>   Filesystem UUID: 80c2f275-ed6b-4ef5-b785-b53bd225ca9e
>   Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912,
>     819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000,
>     23887872
>   Allocating group tables: done                            
>   Writing inode tables: done                            
>   Writing superblocks and filesystem accounting information:
>     mke2fs: Input/output error while writing out and closing file system
>
> I tried repartitioning the stick into  2 x 128 GB  partitions,
> in case it was the sheer size which was the problem, but got the same result.
> The error occured with both sticks, so it doesn't seem to be bad hardware.
> It took  10 h 40 m  to process the  256 GB  part'n on my 2023 desktop machine,
> so trying suggestions cb a rather long-drawn-out affair (smile).
>
> Has anyone else encountered this ?  Does anyone have suggestions ?

Are there kernel error or warning messages when this happens?

-- 
Nuno Silva

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick

[Michael]

[-- Attachment #1: Type: text/plain, Size: 2785 bytes --]

On Saturday 15 February 2025 11:50:23 Greenwich Mean Time Nuno Silva wrote:
> On 2025-02-15, Philip Webb wrote:
> > Recently, I bought  2  new Kingston Exodia  256 GB  USB sticks
> > from Canada Computers, the store in Toronto I've used for  25 yr .
> > With many previous new USB sticks of sizes  <= 128 GB
> > & which came with a VFat filesystem,
> > I simply repartitioned them using Fdisk, which created a Linux partition
> > & then used 'mke2fs' to format them with an Ext2 filesystem.
> > 
> > This time, something has gone wrong :
> >   root:538 ~> mke2fs /dev/sdb1
> >   mke2fs 1.47.0 (5-Feb-2023)
> >   Creating filesystem with 60567296 4k blocks and 15147008 inodes
> >   Filesystem UUID: 80c2f275-ed6b-4ef5-b785-b53bd225ca9e
> >   Superblock backups stored on blocks: 32768, 98304, 163840, 229376,
> >   294912,
> >   
> >     819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424,
> >     20480000, 23887872
> >   
> >   Allocating group tables: done
> >   Writing inode tables: done
> >   
> >   Writing superblocks and filesystem accounting information:
> >     mke2fs: Input/output error while writing out and closing file system
> > 
> > I tried repartitioning the stick into  2 x 128 GB  partitions,
> > in case it was the sheer size which was the problem, but got the same
> > result. The error occured with both sticks, so it doesn't seem to be bad
> > hardware. It took  10 h 40 m  to process the  256 GB  part'n on my 2023
> > desktop machine, so trying suggestions cb a rather long-drawn-out affair
> > (smile).
> > 
> > Has anyone else encountered this ?  Does anyone have suggestions ?
> 
> Are there kernel error or warning messages when this happens?

An ext2fs with 4K block size has a maximum filesystem size limit of 16TiB.  
Your 256GB drive will not experience a formatting problem because of its size.

Formatting a 256GB USB drive, especially if it is a USB 3.0 or later spec, 
should not take hours, but minutes if not seconds.  Assuming there was no 
power cut or interruption to the formatting operation the error has the smell 
of a hardware problem, hence dmesg should reveal if something went wrong with 
the device.

You can try reformatting the USB drive, while keeping an eye on the output of 
'dmesg -W'.

If both of these sticks are behaving the same way, it could be the port on 
your PC which has a problem.  You can try using a different USB port to 
eliminate this causing the formatting failure.

Other than a hardware problem with the device itself, there is the chance of 
counterfeit USB drives, churned out at volume and having a smaller size and 
speed than advertised, or such poor quality flash chips they end up corrupting 
data.  Usually they survive a reformat, at least with FAT, but can fail at any 
point thereafter.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick

[Philip Webb]

250215 Michael wrote:
> On Saturday 15 February 2025 11:50:23 Greenwich Mean Time Nuno Silva wrote:
> > On 2025-02-15, Philip Webb wrote:
> > > Recently, I bought  2  new Kingston Exodia  256 GB  USB sticks
> > > from Canada Computers, the store in Toronto I've used for  25 yr .
> > > With many previous new USB sticks of sizes  <= 128 GB
> > > & which came with a VFat filesystem,
> > > I simply repartitioned them using Fdisk, which created a Linux partition
> > > & then used 'mke2fs' to format them with an Ext2 filesystem.
> > > 
> > > This time, something has gone wrong :
> > >   root:538 ~> mke2fs /dev/sdb1
> > >   mke2fs 1.47.0 (5-Feb-2023)
> > >   Creating filesystem with 60567296 4k blocks and 15147008 inodes
> > >   Filesystem UUID: 80c2f275-ed6b-4ef5-b785-b53bd225ca9e
> > >   Superblock backups stored on blocks: 32768, 98304, 163840, 229376,
> > >   294912,
> > >   
> > >     819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424,
> > >     20480000, 23887872
> > >   
> > >   Allocating group tables: done
> > >   Writing inode tables: done
> > >   
> > >   Writing superblocks and filesystem accounting information:
> > >     mke2fs: Input/output error while writing out and closing file system
> > > 
> > > I tried repartitioning the stick into  2 x 128 GB  partitions,
> > > in case it was the sheer size which was the problem, but got the same
> > > result. The error occured with both sticks, so it doesn't seem to be bad
> > > hardware. It took  10 h 40 m  to process the  256 GB  part'n on my 2023
> > > desktop machine, so trying suggestions cb a rather long-drawn-out affair
> > > (smile).
> > > 
> > > Has anyone else encountered this ?  Does anyone have suggestions ?
> > 
> > Are there kernel error or warning messages when this happens?
> 
> An ext2fs with 4K block size has a maximum filesystem size limit of 16TiB.  
> Your 256GB drive will not experience a formatting problem because of its size.
> 
> Formatting a 256GB USB drive, especially if it is a USB 3.0 or later spec, 
> should not take hours, but minutes if not seconds.

See listing below.  My notes tell me that in many previous cases,
it has taken these rates to format :  2 :  6 min/GB  ; 3 :  1,8 min/GB ;
today, it took  2 h 51 m  to format a  64 GB  partition (mainly inodes).

> Assuming there was no power cut or interruption to the formatting operation,
> the error has the smell of a hardware problem,
> hence dmesg should reveal if something went wrong with the device.
> You can try reformatting the USB drive,
> while keeping an eye on the output of 'dmesg -W'.

Here's the output of the formatting + 'dmesg -W' ;
I used a different port, which is known to behave properly,
+ the other stick now re-partitioned to offer a  64 GB  partition :

root:554 ~> t ; mke2fs /dev/sdb1 ; t
                        2025-02-15   Sat   17.57.46
mke2fs 1.47.0 (5-Feb-2023)
Creating filesystem with 16777216 4k blocks and 4194304 inodes
Filesystem UUID: 93b5ff29-fe5d-48e5-85b0-35b12bee226e
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: mke2fs: Input/output error while writing out and closing file system
                        2025-02-15   Sat   20.48.04

dmesg 250215 21:12 : no messages while writing inode tables ;
  different 3.0 port normally used without problem by scanner ;
  'ls /dev' shows /dev/sdb sdb1 sdb2 sdb3 sdb4 still listed ;
  stick is still in the port

root:635 ~> dmesg -W
[2023143.202399] usb 10-2: USB disconnect, device number 2
[2023143.210193] blk_print_req_error: 716 callbacks suppressed
[2023143.210196] device offline error, dev sdb, sector 95422464 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 2
[2023143.210202] buffer_io_error: 1328 callbacks suppressed
[2023143.210203] Buffer I/O error on dev sdb1, logical block 11927552, lost async page write
[2023143.210218] device offline error, dev sdb, sector 95422472 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2
[2023143.210221] Buffer I/O error on dev sdb1, logical block 11927553, lost async page write
[2023143.210259] device offline error, dev sdb, sector 95684608 op 0x1:(WRITE) flags 0x100000 phys_seg 2 prio class 2
[2023143.210264] Buffer I/O error on dev sdb1, logical block 11960320, lost async page write
[2023143.210269] Buffer I/O error on dev sdb1, logical block 11960321, lost async page write
[2023143.210277] device offline error, dev sdb, sector 95946752 op 0x1:(WRITE) flags 0x100000 phys_seg 2 prio class 2
[2023143.210280] Buffer I/O error on dev sdb1, logical block 11993088, lost async page write
[2023143.210283] Buffer I/O error on dev sdb1, logical block 11993089, lost async page write
[2023143.210302] device offline error, dev sdb, sector 96208896 op 0x1:(WRITE) flags 0x100000 phys_seg 2 prio class 2
[2023143.210305] Buffer I/O error on dev sdb1, logical block 12025856, lost async page write
[2023143.210308] Buffer I/O error on dev sdb1, logical block 12025857, lost async page write
[2023143.210312] device offline error, dev sdb, sector 96471040 op 0x1:(WRITE) flags 0x100000 phys_seg 2 prio class 2
[2023143.210315] Buffer I/O error on dev sdb1, logical block 12058624, lost async page write
[2023143.210342] Buffer I/O error on dev sdb1, logical block 12058625, lost async page write
[2023143.210351] device offline error, dev sdb, sector 96733184 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2
[2023143.210373] device offline error, dev sdb, sector 96733192 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 2
[2023143.210382] device offline error, dev sdb, sector 96995328 op 0x1:(WRITE) flags 0x800 phys_seg 2 prio class 2
[2023143.210401] device offline error, dev sdb, sector 97257472 op 0x1:(WRITE) flags 0x800 phys_seg 2 prio class 2
[2023143.926864] usb 10-2: new SuperSpeed USB device number 3 using xhci_hcd
[2023143.946392] usb 10-2: New USB device found, idVendor=0951, idProduct=1666, bcdDevice= 1.10
[2023143.946397] usb 10-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[2023143.946400] usb 10-2: Product: DataTraveler 3.0
[2023143.946402] usb 10-2: Manufacturer: Kingston
[2023143.946404] usb 10-2: SerialNumber: E0D55EA57410E8B189D80112
[2023143.946819] usb-storage 10-2:1.0: USB Mass Storage device detected
[2023143.946993] scsi host12: usb-storage 10-2:1.0
[2023144.950983] scsi 12:0:0:0: Direct-Access     Kingston DataTraveler 3.0 PMAP PQ: 0 ANSI: 6
[2023144.951249] sd 12:0:0:0: Attached scsi generic sg1 type 0
[2023144.951349] sd 12:0:0:0: [sdb] 484540416 512-byte logical blocks: (248 GB/231 GiB)
[2023144.951913] sd 12:0:0:0: [sdb] Write Protect is off
[2023144.951917] sd 12:0:0:0: [sdb] Mode Sense: 45 00 00 00
[2023144.952478] sd 12:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[2023145.369087]  sdb: sdb1 sdb2 sdb3 sdb4 < >
[2023145.369453] sd 12:0:0:0: [sdb] Attached SCSI removable disk

I can't make sense of the "USB disconnect" at the beginning
nor the "Attached SCSI removable disk" at the end :
I didn't touch anything during the whole process & have no removable disks.

> If both of these sticks are behaving the same way,
> it could be the port on your PC which has a problem.
> You can try using a different USB port
> to eliminate this causing the formatting failure.

I used the port normally used by my scanner with no problems.
The behaviour is the same.

> Other than a hardware problem with the device itself,
> there is the chance of counterfeit USB drives, churned out at volume
> and having a smaller size and speed than advertised
> or such poor quality flash chips they end up corrupting data.
> Usually they survive a reformat, at least with FAT,
> but can fail at any point thereafter.

That's very unlikely for sticks bought from a reputable store.
I've used Canada Computers since 2000 & had a problem only once,
a defective mobo for a new machine, which was replaced by another make.

Can anyone find anything of interest in the logs above ?

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

[gentoo-user] Re: problem formatting new 256 GB USB stick

[Nuno Silva]

On 2025-02-16, Philip Webb wrote:

> 250215 Michael wrote:
>> On Saturday 15 February 2025 11:50:23 Greenwich Mean Time Nuno Silva wrote:
>> > On 2025-02-15, Philip Webb wrote:
>> > > Recently, I bought  2  new Kingston Exodia  256 GB  USB sticks
>> > > from Canada Computers, the store in Toronto I've used for  25 yr .
>> > > With many previous new USB sticks of sizes  <= 128 GB
>> > > & which came with a VFat filesystem,
>> > > I simply repartitioned them using Fdisk, which created a Linux partition
>> > > & then used 'mke2fs' to format them with an Ext2 filesystem.
>> > > 
>> > > This time, something has gone wrong :
>> > >   root:538 ~> mke2fs /dev/sdb1
>> > >   mke2fs 1.47.0 (5-Feb-2023)
[...]
>> > >   Writing superblocks and filesystem accounting information:
>> > >     mke2fs: Input/output error while writing out and closing file system
[...]
>> > > Has anyone else encountered this ?  Does anyone have suggestions ?
>> > 
>> > Are there kernel error or warning messages when this happens?
>> 
>> An ext2fs with 4K block size has a maximum filesystem size limit of 16TiB.  
>> Your 256GB drive will not experience a formatting problem because of its size.
>> 
>> Formatting a 256GB USB drive, especially if it is a USB 3.0 or later spec, 
>> should not take hours, but minutes if not seconds.
>
> See listing below.  My notes tell me that in many previous cases,
> it has taken these rates to format :  2 :  6 min/GB  ; 3 :  1,8 min/GB ;
> today, it took  2 h 51 m  to format a  64 GB  partition (mainly inodes).
>
>> Assuming there was no power cut or interruption to the formatting operation,
>> the error has the smell of a hardware problem,
>> hence dmesg should reveal if something went wrong with the device.
>> You can try reformatting the USB drive,
>> while keeping an eye on the output of 'dmesg -W'.
>
> Here's the output of the formatting + 'dmesg -W' ;
> I used a different port, which is known to behave properly,
> + the other stick now re-partitioned to offer a  64 GB  partition :
>
> root:554 ~> t ; mke2fs /dev/sdb1 ; t
>                         2025-02-15   Sat   17.57.46
> mke2fs 1.47.0 (5-Feb-2023)
> Creating filesystem with 16777216 4k blocks and 4194304 inodes
> Filesystem UUID: 93b5ff29-fe5d-48e5-85b0-35b12bee226e
> Superblock backups stored on blocks: 
>         32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
>         4096000, 7962624, 11239424
>
> Allocating group tables: done                            
> Writing inode tables: done                            
> Writing superblocks and filesystem accounting information: mke2fs: Input/output error while writing out and closing file system
>                         2025-02-15   Sat   20.48.04
>
> dmesg 250215 21:12 : no messages while writing inode tables ;
>   different 3.0 port normally used without problem by scanner ;
>   'ls /dev' shows /dev/sdb sdb1 sdb2 sdb3 sdb4 still listed ;
>   stick is still in the port
>
> root:635 ~> dmesg -W
> [2023143.202399] usb 10-2: USB disconnect, device number 2
> [2023143.210193] blk_print_req_error: 716 callbacks suppressed
> [2023143.210196] device offline error, dev sdb, sector 95422464 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 2
> [2023143.210202] buffer_io_error: 1328 callbacks suppressed
> [2023143.210203] Buffer I/O error on dev sdb1, logical block 11927552, lost async page write
> [2023143.210218] device offline error, dev sdb, sector 95422472 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2
[...]
> [2023143.210401] device offline error, dev sdb, sector 97257472 op 0x1:(WRITE) flags 0x800 phys_seg 2 prio class 2
> [2023143.926864] usb 10-2: new SuperSpeed USB device number 3 using xhci_hcd
> [2023143.946392] usb 10-2: New USB device found, idVendor=0951, idProduct=1666, bcdDevice= 1.10
> [2023143.946397] usb 10-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
> [2023143.946400] usb 10-2: Product: DataTraveler 3.0
> [2023143.946402] usb 10-2: Manufacturer: Kingston
> [2023143.946404] usb 10-2: SerialNumber: E0D55EA57410E8B189D80112
> [2023143.946819] usb-storage 10-2:1.0: USB Mass Storage device detected
> [2023143.946993] scsi host12: usb-storage 10-2:1.0
> [2023144.950983] scsi 12:0:0:0: Direct-Access     Kingston DataTraveler 3.0 PMAP PQ: 0 ANSI: 6
> [2023144.951249] sd 12:0:0:0: Attached scsi generic sg1 type 0
> [2023144.951349] sd 12:0:0:0: [sdb] 484540416 512-byte logical blocks: (248 GB/231 GiB)
> [2023144.951913] sd 12:0:0:0: [sdb] Write Protect is off
> [2023144.951917] sd 12:0:0:0: [sdb] Mode Sense: 45 00 00 00
> [2023144.952478] sd 12:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
> [2023145.369087]  sdb: sdb1 sdb2 sdb3 sdb4 < >
> [2023145.369453] sd 12:0:0:0: [sdb] Attached SCSI removable disk
>
> I can't make sense of the "USB disconnect" at the beginning
> nor the "Attached SCSI removable disk" at the end :
> I didn't touch anything during the whole process & have no removable
> disks.
>
>> If both of these sticks are behaving the same way,
>> it could be the port on your PC which has a problem.
>> You can try using a different USB port
>> to eliminate this causing the formatting failure.
>
> I used the port normally used by my scanner with no problems.
> The behaviour is the same.

The device disconnected and then reconnected, this hopefully means there
is nothing wrong with the flash stick itself, that you just have to
figure out what is disconnecting it: lack of power, bad cable (if
applicable), usb power saving...

I've seen USB3 devices be very unstable when connected directly to some
USB ports. Adding a (USB extension, all of this with USB A) cable solved
the issue on the machine I saw that on. But that's probably not the
case, unless what you're seeing is mostly reproducible but random in how
long it takes to manifest (i.e. time between mkfs and the first
disconnect message in the kernel log).

-- 
Nuno Silva

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick

[Michael]

[-- Attachment #1: Type: text/plain, Size: 6937 bytes --]

On Sunday 16 February 2025 09:08:10 Greenwich Mean Time Nuno Silva wrote:
> On 2025-02-16, Philip Webb wrote:
> > 250215 Michael wrote:

> >> Formatting a 256GB USB drive, especially if it is a USB 3.0 or later
> >> spec, should not take hours, but minutes if not seconds.
> > 
> > See listing below.  My notes tell me that in many previous cases,
> > it has taken these rates to format :  2 :  6 min/GB  ; 3 :  1,8 min/GB ;
> > today, it took  2 h 51 m  to format a  64 GB  partition (mainly inodes).

I just formatted a USB 2.0 8GB stick with mke2fs as a test.  It took 46 
seconds.  Extrapolating for your 64GB partition it should take ~ 6 minutes.  A 
full 256GB drive would take 24.5 minutes.  I expect a drive enjoying USB 3.0 
transfer speeds to take way less than this.


> > root:635 ~> dmesg -W
> > [2023143.202399] usb 10-2: USB disconnect, device number 2
> > [2023143.210193] blk_print_req_error: 716 callbacks suppressed
> > [2023143.210196] device offline error, dev sdb, sector 95422464 op
> > 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 2 [2023143.210202]
> > buffer_io_error: 1328 callbacks suppressed
> > [2023143.210203] Buffer I/O error on dev sdb1, logical block 11927552,
> > lost async page write [2023143.210218] device offline error, dev sdb,
> > sector 95422472 op 0x1:(WRITE) flags 0x100000 phys_seg 1 prio class 2
> [...]
> 
> > [2023143.210401] device offline error, dev sdb, sector 97257472 op
> > 0x1:(WRITE) flags 0x800 phys_seg 2 prio class 2 [2023143.926864] usb
> > 10-2: new SuperSpeed USB device number 3 using xhci_hcd [2023143.946392]
> > usb 10-2: New USB device found, idVendor=0951, idProduct=1666, bcdDevice=
> > 1.10 [2023143.946397] usb 10-2: New USB device strings: Mfr=1, Product=2,
> > SerialNumber=3 [2023143.946400] usb 10-2: Product: DataTraveler 3.0
> > [2023143.946402] usb 10-2: Manufacturer: Kingston
> > [2023143.946404] usb 10-2: SerialNumber: E0D55EA57410E8B189D80112
> > [2023143.946819] usb-storage 10-2:1.0: USB Mass Storage device detected
> > [2023143.946993] scsi host12: usb-storage 10-2:1.0
> > [2023144.950983] scsi 12:0:0:0: Direct-Access     Kingston DataTraveler
> > 3.0 PMAP PQ: 0 ANSI: 6 [2023144.951249] sd 12:0:0:0: Attached scsi
> > generic sg1 type 0
> > [2023144.951349] sd 12:0:0:0: [sdb] 484540416 512-byte logical blocks:
> > (248 GB/231 GiB) [2023144.951913] sd 12:0:0:0: [sdb] Write Protect is off
> > [2023144.951917] sd 12:0:0:0: [sdb] Mode Sense: 45 00 00 00
> > [2023144.952478] sd 12:0:0:0: [sdb] Write cache: disabled, read cache:
> > enabled, doesn't support DPO or FUA [2023145.369087]  sdb: sdb1 sdb2 sdb3
> > sdb4 < >
> > [2023145.369453] sd 12:0:0:0: [sdb] Attached SCSI removable disk
> > 
> > I can't make sense of the "USB disconnect" at the beginning

This is the message you get when you physically unplug a USB drive.


> > nor the "Attached SCSI removable disk" at the end :

This is the message you get when the device is powered up, detected by the 
kernel and the filesystem is then being accessed.  From this point on the 
device can be read from and written to.


> > I didn't touch anything during the whole process & have no removable
> > disks.

Needless to say, if you did not touch the USB stick when the disconnection 
message was recorded something disconnected (electrically) the device.


> >> If both of these sticks are behaving the same way,
> >> it could be the port on your PC which has a problem.
> >> You can try using a different USB port
> >> to eliminate this causing the formatting failure.
> > 
> > I used the port normally used by my scanner with no problems.
> > The behaviour is the same.

Hmm ... this points to the USB sticks, rather than the ports on your device.


> The device disconnected and then reconnected, this hopefully means there
> is nothing wrong with the flash stick itself, that you just have to
> figure out what is disconnecting it: lack of power, bad cable (if
> applicable), usb power saving...

Wouldn't the same problem manifest with other USB sticks?  That said, I have 
an old USB stick which fails to connect properly if I push it in all the way 
in a port.  I have to push it in, then imperceptibly try to pull it out to 
make good electrical contact with the port.  I've blamed corrosion on the 
electrical strips inside the USB connector - it had been dropped in a mug of 
coffee at some point!  Philip's USB sticks are brand new.


> I've seen USB3 devices be very unstable when connected directly to some
> USB ports. Adding a (USB extension, all of this with USB A) cable solved
> the issue on the machine I saw that on. But that's probably not the
> case, unless what you're seeing is mostly reproducible but random in how
> long it takes to manifest (i.e. time between mkfs and the first
> disconnect message in the kernel log).

I've had a similar problem with a new USB 3.0 stick.  I can't recall the make/
model, but it was not some cheap knock-off.  The speeds were glacial, rsync 
would fail sooner or later when copying data over, disconnections were shown 
on dmesg and the stick would become really hot to touch.  I tried reformatting 
it with FAT, ext2 and finally with F2FS, with improving performance each time 
due to the lighter load placed by the filesystem type on the device.  However, 
the disconnections and failures continued until I returned it as a faulty 
product.  Unlike the more expensive USB SSD drives, the onboard controller of 
the lower cost USB flash sticks tend to be cheap and unsophisticated, with a 
higher probability of early failure.


> >> Other than a hardware problem with the device itself,
> >> there is the chance of counterfeit USB drives, churned out at volume
> >> and having a smaller size and speed than advertised
> >> or such poor quality flash chips they end up corrupting data.
> >> Usually they survive a reformat, at least with FAT,
> >> but can fail at any point thereafter.
> > 
> > That's very unlikely for sticks bought from a reputable store.
> > I've used Canada Computers since 2000 & had a problem only once,
> > a defective mobo for a new machine, which was replaced by another make.

"Reputable store" these days implies they will not try to rob you, they have 
some rudimentary technical knowledge about their products and you can return 
faulty products for a replacement/refund.  Other than that they are all box-
shifters of products typically manufactured in the Far East at minimum cost 
and invariably with very little quality control.

Regarding the USB stick marketed as Kingston Exodia 256 GB, the dmesg shown 
brand and model "idVendor=0951, idProduct=1666" helps identify it as being the 
same with the older Kingston DataTraveler 100 G3/G4/SE9 G2/50:

https://devicehunt.com/view/type/usb/vendor/0951/device/1666

If you intend to return it to the store, first reformat it with FAT and do not 
mention your ext2 experience, in case they try to blame that as the cause of 
your disconnections.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick

[Wols Lists]

On 16/02/2025 13:41, Michael wrote:
>>> nor the "Attached SCSI removable disk" at the end :

> This is the message you get when the device is powered up, detected by the
> kernel and the filesystem is then being accessed.  From this point on the
> device can be read from and written to.

So (and this was my instant reaction on first reading the thread), is 
that you might be getting messages that you should expect but naively 
didn't.

You expect a message to say "detected USB drive". You expect a message 
to say "accessing file system(s)".

Except you then ran fdisk and deleted the MBR/GPT! So you're going to 
get a bunch of (possibly unexpected) messages about filesystems going 
away and reappearing ...

I'm not sure what else is going to happen, but there might be more 
"unexpected but obvious in hindsight" messages flying around.

The other thing. USB. Does all sorts of weird things. Just because your 
USB stick is USB3 doesn't mean you're going to get USB3 performance from 
it. Much as I don't like gaming mobos, you can reasonably assume 
manufacturers aren't gaming the system here - there's too many 
knowledgable people who will catch them out ...

But a possible scenario is the mobo has a single USB3 hub (probable, it 
can handle 128 devices) wired into the mobo, with a USB2 hub wired into 
the USB3 hub. I doubt it has a USB1 hub ... But some cheap USB hubs run 
at the speed of the slowest device - if your scanner is plugged in, 
live, and advertises itself as USB2, it could downgrade a cheap USB3 hub 
to USB2! Ouch! Or if it's just plugged into a USB2 port and holding the 
USB2 hub live!

Completely different, but I think I've just debugged a pain point at 
home. I've got a telco-supplied mesh network, which is crap. So I run 
ethernet-over-power, which WAS crap. Dunno why. Buy a tp-link router, 
configure it as an access point over CAT-5, plug the master 
ethernet-over-power into the access point not the telco hub, and bingo, 
everything is working so much better! Your trouble with these sticks is 
probably the same, make a couple of changes that as far as you can see 
should make no difference whatsoever, but they make a massive 
improvement instead ...

Cheers,
Wol

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

Am Sun, Feb 16, 2025 at 01:41:10PM +0000 schrieb Michael:
> On Sunday 16 February 2025 09:08:10 Greenwich Mean Time Nuno Silva wrote:
> > On 2025-02-16, Philip Webb wrote:
> > > 250215 Michael wrote:
> 
> > >> Formatting a 256GB USB drive, especially if it is a USB 3.0 or later
> > >> spec, should not take hours, but minutes if not seconds.
> > > 
> > > See listing below.  My notes tell me that in many previous cases,
> > > it has taken these rates to format :  2 :  6 min/GB  ; 3 :  1,8 min/GB ;
> > > today, it took  2 h 51 m  to format a  64 GB  partition (mainly inodes).
> 
> I just formatted a USB 2.0 8GB stick with mke2fs as a test.  It took 46 
> seconds.  Extrapolating for your 64GB partition it should take ~ 6 minutes.  A 
> full 256GB drive would take 24.5 minutes.  I expect a drive enjoying USB 3.0 
> transfer speeds to take way less than this.

Your expectations have some vague assumptions:
- the stick achieves full speed of its USB protocol
- it keeps that throughput over time
- the data written grows linearly with the FS size

USB sticks often have a bad thermal design and throttle down once they get 
hot. And hot they get, especially the high-speed ones.

If you are curious, use a program like dool or glances to observe the 
device’s actual write throughput during formatting. htop can show this, too, 
but you first need to add a display meter in its config.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

For Sale:
  Parachute.  Used once.  Never opened.  Slightly stained.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1870 bytes --]

On Monday 17 February 2025 23:12:59 Greenwich Mean Time Frank Steinmetzger 
wrote:
> Am Sun, Feb 16, 2025 at 01:41:10PM +0000 schrieb Michael:

> > I just formatted a USB 2.0 8GB stick with mke2fs as a test.  It took 46
> > seconds.  Extrapolating for your 64GB partition it should take ~ 6
> > minutes.  A full 256GB drive would take 24.5 minutes.  I expect a drive
> > enjoying USB 3.0 transfer speeds to take way less than this.
> 
> Your expectations have some vague assumptions:

Yes, I was just trying to point loosely at the order of magnitude as an 
indication of something being faulty in the USB Philip was trying to format.

> - the stick achieves full speed of its USB protocol

I haven't yet found a USB stick which reaches the maximum USB protocol speeds.  
The old USB 2.0 stick I used will not go above 11-12MBps during large writes 
and around twice as fast on sustained reads, well below the advertised USB 2.0 
speed of 60MBps.

> - it keeps that throughput over time

Once buffers are saturated write operations will settle at what the device can 
achieve.  I have observed writes slow down when the empty space left is 
getting low. 

> - the data written grows linearly with the FS size

Usually larger USB drives of the same brand/model tend to have slightly better 
speeds - at least this is what the data sheet for the USB drive Philip bought 
claims.


> USB sticks often have a bad thermal design and throttle down once they get
> hot. And hot they get, especially the high-speed ones.
> 
> If you are curious, use a program like dool or glances to observe the
> device’s actual write throughput during formatting. htop can show this, too,
> but you first need to add a display meter in its config.

I use gkrellms on GUI, and 'watch -d vmstat -d -S M' when on a console, but 
I've also used htop.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]

Am Tue, Feb 18, 2025 at 11:53:48AM +0000 schrieb Michael:

> > Your expectations have some vague assumptions:
> 
> Yes, I was just trying to point loosely at the order of magnitude as an 
> indication of something being faulty in the USB Philip was trying to format.
> 
> > - the stick achieves full speed of its USB protocol
> 
> I haven't yet found a USB stick which reaches the maximum USB protocol speeds.  
> The old USB 2.0 stick I used will not go above 11-12MBps during large writes 
> and around twice as fast on sustained reads, well below the advertised USB 2.0 
> speed of 60MBps.

I’ve never seen more than 40 MB/s, also with USB 3.0 HDDs attached to a 2.0 
port.

> > - it keeps that throughput over time
> 
> Once buffers are saturated write operations will settle at what the device can 
> achieve.  I have observed writes slow down when the empty space left is 
> getting low. 

Which buffer? Sticks don’t generally have one, let a lone an equivalent to 
an SSD’s SLC cache. I guess you mean the host’s cache?

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

For security reasons, all text in this mail is double-rot13 encrypted.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick

[Michael]

[-- Attachment #1: Type: text/plain, Size: 912 bytes --]

On Wednesday 19 February 2025 12:10:59 Greenwich Mean Time Frank Steinmetzger 
wrote:
> Am Tue, Feb 18, 2025 at 11:53:48AM +0000 schrieb Michael:

> > Once buffers are saturated write operations will settle at what the device
> > can achieve.  I have observed writes slow down when the empty space left
> > is getting low.
> 
> Which buffer? Sticks don’t generally have one, let a lone an equivalent to
> an SSD’s SLC cache. I guess you mean the host’s cache?

Yes.  I'm talking about dumb USB sticks with no onboard DRAM.  You can see 
initially a small duration upsurge in speed, before the data transfer settles 
to steady state.  After the command completes (cp , dd, etc.) and exits in the 
terminal, there is still data shown being transferred for some seconds until 
the buffer empties.

When carrying out successive tests you have to empty the buffers to obtain 
accurate results.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[gentoo-user] problem formatting new 256 GB USB stick : more info

[Philip Webb]

I successfully formatted one of the partitions which failed with Ext2 as Vfat.
I was able to mount it, create a file with words in it,
save it, list it via 'ls', browse it & then delete it, all using Gentoo.
This suggests that the problem isn't due to defective hardware,
but is somewhere in 'mke2fs' or related material.

Any observations are very welcome.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

[gentoo-user] Re: problem formatting new 256 GB USB stick : more info

[Nuno Silva]

On 2025-02-16, Philip Webb wrote:

> I successfully formatted one of the partitions which failed with Ext2 as Vfat.
> I was able to mount it, create a file with words in it,
> save it, list it via 'ls', browse it & then delete it, all using Gentoo.
> This suggests that the problem isn't due to defective hardware,
> but is somewhere in 'mke2fs' or related material.
>
> Any observations are very welcome.

I'd say this is very unlikely, and that you should test and investigate
more, lest it have some issue that could lead to data loss later.

-- 
Nuno Silva

Re: [gentoo-user] problem formatting new 256 GB USB stick : more info

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1412 bytes --]

On Sunday 16 February 2025 22:57:29 Greenwich Mean Time Philip Webb wrote:
> I successfully formatted one of the partitions which failed with Ext2 as
> Vfat. I was able to mount it, create a file with words in it,
> save it, list it via 'ls', browse it & then delete it, all using Gentoo.
> This suggests that the problem isn't due to defective hardware,
> but is somewhere in 'mke2fs' or related material.
> 
> Any observations are very welcome.

A USB drive which disconnects itself without interference by the user does not 
indicate a problem caused by any filesystem in and of itself.

However, a poor quality or buggy USB flash controller which drops the 
connection because it is overloaded by data, could produce all sort of random 
failures.

I am no filesystem expert, but my thinking is an ext2 filesystem will try to 
commit more data to a block device than FAT does while formatted.  Ext2 will 
write in many different blocks across the partition by creating backups of its 
superblock, inode bitmaps, inode tables, etc.  If a USB drive cannot cope with 
this relatively simple transaction of committing to flash cells the structure 
of a filesystem, then it must have bigger problems.

Instead of writing and deleting a simple text file, I would try to stress test 
the drive by copying over a more demanding workload in order to see if this 
succeeds without dmesg coming up with any more errors.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] problem formatting new 256 GB USB stick : more info

[Philip Webb]

250217 Michael wrote:
> On Sunday 16 February 2025 22:57:29 Greenwich Mean Time Philip Webb wrote:
>> I successfully formatted one of the partitions which failed with Ext2 as
>> Vfat. I was able to mount it, create a file with words in it,
>> save it, list it via 'ls', browse it & then delete it, all using Gentoo.
>> This suggests that the problem isn't due to defective hardware,
>> but is somewhere in 'mke2fs' or related material.
>> Any observations are very welcome.
> A USB drive which disconnects itself without interference by the user
> does not indicate a problem caused by any filesystem in and of itself.
> However, a poor quality or buggy USB flash controller
> which drops the connection because it is overloaded by data
> could produce all sort of random failures.  I am no filesystem expert,
> but my thinking is an ext2 filesystem will try to commit more data
> to a block device than FAT does while formatted.
> Ext2 will write in many different blocks across the partition
> by creating backups of its superblock, inode bitmaps, inode tables, etc.
> If a USB drive cannot cope with this relatively simple transaction
> of committing to flash cells the structure of a filesystem,
> then it must have bigger problems.  Instead of writing and deleting
> a simple text file, I would try to stress test the drive
> by copying over a more demanding workload
> to see if this succeeds without dmesg coming up with any more errors.

I do hear you + Nuno clearly, but my own experience is
that repeatable errors are the result of software problems
-- incl eg omitted flags or inadequate config files --
& random errors are caused by faulty hardware.
This problem is repeatable with 'mke2fs' : do I need a flag or config ?
-- that cb something which has changed with the latest  256 GB  sticks.

I'm currently testing a 2.0/1.1 port to write an Ext2 file system.
If that makes no difference, I'll see if I can do your stress test above
after recreating a FAT file system on one of the partitions.
Another test mb to create a very small partition -- eg  5 GB  --
& see if 'mke2fs' wb able to format that without disconnecting.
I'll report results when I have them (smile).

The sticks were delivered, so I can't easily return them,
& in any case we've had  50 cm  snow dumped on us in the last few days.
If a Linux file system really is unachievable,
I can format the sticks as FAT, which sb adequate for simple archiving.

Thanks to both of you for your advice so far.
Comments from others are welcome too.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

Re: [gentoo-user] problem formatting new 256 GB USB stick : more info

[Philip Webb]

250216 Philip Webb wrote:
> I'm currently testing a 2.0/1.1 port to write an Ext2 file system.

it succeeded ! -- results below :

  root:668 ~> t ; mke2fs /dev/sdb1 ; t
    2025-02-16   Sun   22.21.11
  mke2fs 1.47.0 (5-Feb-2023)
  Creating filesystem with 16777216 4k blocks and 4194304 inodes
  Filesystem UUID: c5de9632-b97e-4b9c-bfd5-d5c5af785d48
  Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
    4096000, 7962624, 11239424
  Allocating group tables: done                            
  Writing inode tables: done                            
  Writing superblocks and filesystem accounting information: done   
    2025-02-17   Mon   02.36.25

as cb seen, it took  4 h 15 m .
there are differences in output from 'dmesg -W' :

  root:502 ~> dmesg -W
  [2125533.280237] usb 5-6.2: reset high-speed USB device number 6 using xhci_hcd
  [2126092.734294] wlp5s0: disconnect from AP 7a:ff:7b:47:c8:7f for new auth to 68:ff:7b:47:c9:13
  [2126092.829576] wlp5s0: authenticate with 68:ff:7b:47:c9:13
  [2126092.859893] wlp5s0: send auth to 68:ff:7b:47:c9:13 (try 1/3)
  [2126092.888971] wlp5s0: authenticated
  [2126092.889223] wlp5s0: associate with 68:ff:7b:47:c9:13 (try 1/3)
  [2126092.901777] wlp5s0: RX ReassocResp from 68:ff:7b:47:c9:13 (capab=0x1511 status=0 aid=1)
  [2126092.926490] wlp5s0: associated
  [2126092.926513] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 68:ff:7b:47:c9:13
  [2126171.655896] wlp5s0: disconnect from AP 68:ff:7b:47:c9:13 for new auth to 7a:ff:7b:47:c8:7f
  [2126171.735470] wlp5s0: authenticate with 7a:ff:7b:47:c8:7f
  [2126171.765556] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 1/3)
  [2126171.871847] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 2/3)
  [2126171.875511] wlp5s0: authenticated
  [2126171.876841] wlp5s0: associate with 7a:ff:7b:47:c8:7f (try 1/3)
  [2126171.889248] wlp5s0: RX ReassocResp from 7a:ff:7b:47:c8:7f (capab=0x1511 status=0 aid=1)
  [2126171.914432] wlp5s0: associated
  [2126171.914464] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 7a:ff:7b:47:c8:7f
  [2128215.129375] wlp5s0: disconnect from AP 7a:ff:7b:47:c8:7f for new auth to 68:ff:7b:47:c9:13
  [2128215.229697] wlp5s0: authenticate with 68:ff:7b:47:c9:13
  [2128215.259597] wlp5s0: send auth to 68:ff:7b:47:c9:13 (try 1/3)
  [2128215.288570] wlp5s0: authenticated
  [2128215.289300] wlp5s0: associate with 68:ff:7b:47:c9:13 (try 1/3)
  [2128215.301360] wlp5s0: RX ReassocResp from 68:ff:7b:47:c9:13 (capab=0x1511 status=0 aid=1)
  [2128215.326165] wlp5s0: associated
  [2128215.334629] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 68:ff:7b:47:c9:13
  [2128230.472949] wlp5s0: disconnect from AP 68:ff:7b:47:c9:13 for new auth to 7a:ff:7b:47:c8:7f
  [2128230.548458] wlp5s0: authenticate with 7a:ff:7b:47:c8:7f
  [2128230.577576] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 1/3)
  [2128230.681826] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 2/3)
  [2128230.684849] wlp5s0: authenticated
  [2128230.685838] wlp5s0: associate with 7a:ff:7b:47:c8:7f (try 1/3)
  [2128230.697992] wlp5s0: RX ReassocResp from 7a:ff:7b:47:c8:7f (capab=0x1511 status=0 aid=1)
  [2128230.723132] wlp5s0: associated
  [2128230.723154] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 7a:ff:7b:47:c8:7f
  [2129066.061800] wlp5s0: disconnect from AP 7a:ff:7b:47:c8:7f for new auth to 68:ff:7b:47:c9:13
  [2129066.140447] wlp5s0: authenticate with 68:ff:7b:47:c9:13
  [2129066.169621] wlp5s0: send auth to 68:ff:7b:47:c9:13 (try 1/3)
  [2129066.198602] wlp5s0: authenticated
  [2129066.199727] wlp5s0: associate with 68:ff:7b:47:c9:13 (try 1/3)
  [2129066.211388] wlp5s0: RX ReassocResp from 68:ff:7b:47:c9:13 (capab=0x1511 status=0 aid=1)
  [2129066.237270] wlp5s0: associated
  [2129066.265480] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 68:ff:7b:47:c9:13
  [2129068.686728] wlp5s0: disconnect from AP 68:ff:7b:47:c9:13 for new auth to 7a:ff:7b:47:c8:7f
  [2129068.768225] wlp5s0: authenticate with 7a:ff:7b:47:c8:7f
  [2129068.798339] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 1/3)
  [2129068.904663] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 2/3)
  [2129068.907726] wlp5s0: authenticated
  [2129068.908705] wlp5s0: associate with 7a:ff:7b:47:c8:7f (try 1/3)
  [2129068.920407] wlp5s0: RX ReassocResp from 7a:ff:7b:47:c8:7f (capab=0x1511 status=0 aid=1)
  [2129068.945293] wlp5s0: associated
  [2129068.945703] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 7a:ff:7b:47:c8:7f
  [2130027.166965] wlp5s0: disconnect from AP 7a:ff:7b:47:c8:7f for new auth to 68:ff:7b:47:c9:13
  [2130027.265257] wlp5s0: authenticate with 68:ff:7b:47:c9:13
  [2130027.295129] wlp5s0: send auth to 68:ff:7b:47:c9:13 (try 1/3)
  [2130027.324148] wlp5s0: authenticated
  [2130027.324894] wlp5s0: associate with 68:ff:7b:47:c9:13 (try 1/3)
  [2130027.337042] wlp5s0: RX ReassocResp from 68:ff:7b:47:c9:13 (capab=0x1511 status=0 aid=1)
  [2130027.361233] wlp5s0: associated
  [2130027.369658] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 68:ff:7b:47:c9:13
  [2130029.835939] wlp5s0: disconnect from AP 68:ff:7b:47:c9:13 for new auth to 7a:ff:7b:47:c8:7f
  [2130029.930077] wlp5s0: authenticate with 7a:ff:7b:47:c8:7f
  [2130029.960635] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 1/3)
  [2130030.067841] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 2/3)
  [2130030.070895] wlp5s0: authenticated
  [2130030.071832] wlp5s0: associate with 7a:ff:7b:47:c8:7f (try 1/3)
  [2130030.084165] wlp5s0: RX ReassocResp from 7a:ff:7b:47:c8:7f (capab=0x1511 status=0 aid=1)
  [2130030.108940] wlp5s0: associated
  [2130030.148960] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 7a:ff:7b:47:c8:7f
  [2132136.363825] wlp5s0: disconnect from AP 7a:ff:7b:47:c8:7f for new auth to 68:ff:7b:47:c9:13
  [2132136.456967] wlp5s0: authenticate with 68:ff:7b:47:c9:13
  [2132136.486972] wlp5s0: send auth to 68:ff:7b:47:c9:13 (try 1/3)
  [2132136.516000] wlp5s0: authenticated
  [2132136.516728] wlp5s0: associate with 68:ff:7b:47:c9:13 (try 1/3)
  [2132136.529204] wlp5s0: RX ReassocResp from 68:ff:7b:47:c9:13 (capab=0x1511 status=0 aid=1)
  [2132136.554633] wlp5s0: associated
  [2132136.663025] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 68:ff:7b:47:c9:13
  [2132139.033739] wlp5s0: disconnect from AP 68:ff:7b:47:c9:13 for new auth to 7a:ff:7b:47:c8:7f
  [2132139.124569] wlp5s0: authenticate with 7a:ff:7b:47:c8:7f
  [2132139.154945] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 1/3)
  [2132139.260645] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 2/3)
  [2132139.263840] wlp5s0: authenticated
  [2132139.264645] wlp5s0: associate with 7a:ff:7b:47:c8:7f (try 1/3)
  [2132139.277129] wlp5s0: RX ReassocResp from 7a:ff:7b:47:c8:7f (capab=0x1511 status=0 aid=1)
  [2132139.302159] wlp5s0: associated
  [2132139.331597] wlp5s0: Limiting TX power to 30 (30 - 0) dBm as advertised by 7a:ff:7b:47:c8:7f
  [2135034.469269] EXT4-fs (sdb1): mounting ext2 file system using the ext4 subsystem
  [2135043.577142] EXT4-fs (sdb1): mounted filesystem c5de9632-b97e-4b9c-bfd5-d5c5af785d48 r/w without journal. Quota mode: none.
  [2135217.544967] EXT4-fs (sdb1): unmounting filesystem c5de9632-b97e-4b9c-bfd5-d5c5af785d48.

This seems to narrow the problem down to the USB 3.2 performance,
which well mb affected by inferior devices on the stick.
I was able to mount the partition, edit a file & browse it,
but re-actions to commands are very slow.
I'll try to format the other partitions on both sticks,
tho' a final verdict will take some time (wry grin).

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

Re: [gentoo-user] problem formatting new 256 GB USB stick : more info

[Michael]

[-- Attachment #1: Type: text/plain, Size: 4441 bytes --]

On Monday 17 February 2025 09:18:45 Greenwich Mean Time Philip Webb wrote:
> 250216 Philip Webb wrote:
> > I'm currently testing a 2.0/1.1 port to write an Ext2 file system.
> 
> it succeeded ! -- results below :
> 
>   root:668 ~> t ; mke2fs /dev/sdb1 ; t
>     2025-02-16   Sun   22.21.11
>   mke2fs 1.47.0 (5-Feb-2023)
>   Creating filesystem with 16777216 4k blocks and 4194304 inodes
>   Filesystem UUID: c5de9632-b97e-4b9c-bfd5-d5c5af785d48
>   Superblock backups stored on blocks:
>     32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
>     4096000, 7962624, 11239424
>   Allocating group tables: done
>   Writing inode tables: done
>   Writing superblocks and filesystem accounting information: done
>     2025-02-17   Mon   02.36.25
> 
> as cb seen, it took  4 h 15 m .

This is an excessively long time for a USB 2.0 port, but thankfully the dmesg 
output no longer showed any disconnections.


> there are differences in output from 'dmesg -W' :
> 
>   root:502 ~> dmesg -W
>   [2125533.280237] usb 5-6.2: reset high-speed USB device number 6 using
> xhci_hcd [2126092.734294] wlp5s0: disconnect from AP 7a:ff:7b:47:c8:7f for
> new auth to 68:ff:7b:47:c9:13 [2126092.829576] wlp5s0: authenticate with
> 68:ff:7b:47:c9:13
>   [2126092.859893] wlp5s0: send auth to 68:ff:7b:47:c9:13 (try 1/3)
[snip ...]

> wlp5s0: authenticate with 7a:ff:7b:47:c8:7f
>   [2132139.154945] wlp5s0: send auth to 7a:ff:7b:47:c8:7f (try 1/3)
[snip ...]

[NOTE: Your wireless appears to be hunting for different AP BSSIDs to connect 
to.  You could stop it doing this and interrupting your internet connection, 
by selecting the AP BSSID with the stronger signal (larger SNR).  Set this in 
your wpa_supplicant.conf to stop it roaming.]


> [2135034.469269] EXT4-fs (sdb1): mounting
> ext2 file system using the ext4 subsystem [2135043.577142] EXT4-fs (sdb1):
> mounted filesystem c5de9632-b97e-4b9c-bfd5-d5c5af785d48 r/w without
> journal. Quota mode: none. 
> [2135217.544967] EXT4-fs (sdb1): unmounting
> filesystem c5de9632-b97e-4b9c-bfd5-d5c5af785d48.
> 
> This seems to narrow the problem down to the USB 3.2 performance,
> which well mb affected by inferior devices on the stick.

Barring some temporary dust/debris in the USB port or USB stick's electrical 
contacts, in my mind the flash controller on these sticks seems highly 
suspect.  I would not trust them with my data, unless I was transferring files 
from one device to another and data loss was not an issue.

USB sticks are manufactured rather cheaply today and sadly their reliability 
leaves much to be desired.  I recall reading somewhere, the flash controller 
on many USB sticks performs no (meaningful) wear levelling at all.


> I was able to mount the partition, edit a file & browse it,
> but re-actions to commands are very slow.
> I'll try to format the other partitions on both sticks,
> tho' a final verdict will take some time (wry grin).

Instead of FAT or ext2 you may want you try formatting with f2fs, which was 
specifically designed for NAND flash devices, or if you want to have 
compatibility with MSWindows OS you can use exfat.  Both are superior to FAT 
and F2FS in particular was designed for NAND flash storage.  I have found F2FS 
to be faster and more reliable than alternatives on USB sticks.  The only 
caution I have taken is to always access it with the same or more recent 
kernels compared to the one I formatted with.

For exfat you'll need to set in your kernel:

~ $ grep EXFAT /usr/src/linux/.config
# DOS/FAT/EXFAT/NT Filesystems
CONFIG_EXFAT_FS=m
CONFIG_EXFAT_DEFAULT_IOCHARSET="utf8"
# end of DOS/FAT/EXFAT/NT Filesystems

For F2FS:

~ $ grep F2FS /usr/src/linux/.config
CONFIG_F2FS_FS=y
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
CONFIG_F2FS_FS_SECURITY=y
# CONFIG_F2FS_CHECK_FS is not set
# CONFIG_F2FS_FAULT_INJECTION is not set
CONFIG_F2FS_FS_COMPRESSION=y
CONFIG_F2FS_FS_LZO=y
CONFIG_F2FS_FS_LZORLE=y
CONFIG_F2FS_FS_LZ4=y
CONFIG_F2FS_FS_LZ4HC=y
CONFIG_F2FS_FS_ZSTD=y
CONFIG_F2FS_IOSTAT=y
# CONFIG_F2FS_UNFAIR_RWSEM is not set

For storage of (many) text and other compressible files you'd probably want to 
give some forethought to deploying compression, which will further reduce the 
load on the stick.

NOTE: Read up on F2FS and exFAT before you used in production them as there 
are some warnings/considerations specific to them, which may make them 
unsuitable for your use cases.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] problem formatting new 256 GB USB stick : more info

[Stroller]

> On 17 Feb 2025, at 12:02, Michael <confabulate@kintzios.com> wrote:
>> 
>> as cb seen, it took  4 h 15 m .
> 
> This is an excessively long time for a USB 2.0 port, but thankfully the dmesg 
> output no longer showed any disconnections.

The poor wee flash controller had to keep erasing the drive's 16MB of flash so many times so it could continue writing the file. 

Re: [gentoo-user] problem formatting new 256 GB USB stick : more info

[Wols Lists]

On 17/02/2025 03:43, Philip Webb wrote:
> The sticks were delivered, so I can't easily return them,
> & in any case we've had  50 cm  snow dumped on us in the last few days.
> If a Linux file system really is unachievable,
> I can format the sticks as FAT, which sb adequate for simple archiving.

Don't you have "fit for purpose"? In the UK if the sticks won't reformat 
to ext2 or whatever you want them for, you'd just return them as not fit 
/ faulty, and consumer protection law kicks in to say you're due a full 
refund.

And if they say "it was provided as FAT, you shouldn't have reformatted 
it", well where did they say that when you bought it, and lots of uses 
(TVs etc) do advise you to reformat sticks.

Plus of course, as was mentioned elsewhere, if it can't cope with a data 
load of reformatting, how do you know it's going to cope with other, 
different data loads?

Cheers,
Wol

[gentoo-user] Re: problem formatting new 256 GB USB stick : more info

[Grant Edwards]

On 2025-02-17, Philip Webb <purslow@ca.inter.net> wrote:

> The sticks were delivered, so I can't easily return them,
> & in any case we've had  50 cm  snow dumped on us in the last few days.
> If a Linux file system really is unachievable,

Sounds to me like the USB flash drives might be fakes.  They might
have only a small fraction of the advertixed space, and the controller
chip firmware has been fudged to pretend there's 256GB.  As long as
you only use a small portion of the "pretend" space and follow access
patterns the match the controller's faking algorithm, the flash drives
will "work".

> I can format the sticks as FAT, which sb adequate for simple
> archiving.

I'd be very, very careful about that.  If you can't reformat them with
a different filesystem, I wouldn't trust that writing large amounts of
data to them will work regardless of the filesystem.

I would only use them for archiving if you do multiple verify passes
on _everything_ after you've done a backup.

--
Grant

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : more info

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1395 bytes --]

On Monday 17 February 2025 17:16:51 Greenwich Mean Time Grant Edwards wrote:
> On 2025-02-17, Philip Webb <purslow@ca.inter.net> wrote:
> > The sticks were delivered, so I can't easily return them,
> > & in any case we've had  50 cm  snow dumped on us in the last few days.
> > If a Linux file system really is unachievable,
> 
> Sounds to me like the USB flash drives might be fakes.  They might
> have only a small fraction of the advertixed space, and the controller
> chip firmware has been fudged to pretend there's 256GB.  As long as
> you only use a small portion of the "pretend" space and follow access
> patterns the match the controller's faking algorithm, the flash drives
> will "work".
> 
> > I can format the sticks as FAT, which sb adequate for simple
> > archiving.
> 
> I'd be very, very careful about that.  If you can't reformat them with
> a different filesystem, I wouldn't trust that writing large amounts of
> data to them will work regardless of the filesystem.
> 
> I would only use them for archiving if you do multiple verify passes
> on _everything_ after you've done a backup.
> 
> --
> Grant

It is worth mentioning the sys-block/f3 package (Fight Flash Fraud), which is 
in portage and can test a USB flash disk to discover if it is fake.  Besides 
the slower f3write and f3read, the f3probe command will only take a few 
minutes and confirm the available space.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : f3

[Philip Webb]

250217 Michael wrote:
> It is worth mentioning the sys-block/f3 package (Fight Flash Fraud),
> which is in Portage and can test a USB flash disk to discover if it is fake.
> Besides the slower f3write and f3read, the f3probe command
> will only take a few minutes and confirm the available space.

Thanks ! -- 'f3write' + 'f3read' refused, as they expect a directory ;
'f3probe' need it to be compiled with 'USE="extra"', but does work :

root:705 ~> f3probe /dev/sdb
F3 probe 8.0
Copyright (C) 2010 Digirati Internet LTDA.
This is free software; see the source for copying conditions.

WARNING: Probing normally takes from a few seconds to 15 minutes, but
         it can take longer. Please be patient.

Probe finished, recovering blocks... Done

Good news: The device `/dev/sdb' is the real thing

Device geometry:
                 *Usable* size: 231.05 GB (484540416 blocks)
                Announced size: 231.05 GB (484540416 blocks)
                        Module: 256.00 GB (2^38 Bytes)
        Approximate cache size: 0.00 Byte (0 blocks), need-reset=no
           Physical block size: 512.00 Byte (2^9 Bytes)

Probe time: 5'29"
root:706 ~> f3probe /dev/sdb
F3 probe 8.0
Copyright (C) 2010 Digirati Internet LTDA.
This is free software; see the source for copying conditions.

WARNING: Probing normally takes from a few seconds to 15 minutes, but
         it can take longer. Please be patient.

Probe finished, recovering blocks... Done

Good news: The device `/dev/sdb' is the real thing

Device geometry:
                 *Usable* size: 231.05 GB (484540416 blocks)
                Announced size: 231.05 GB (484540416 blocks)
                        Module: 256.00 GB (2^38 Bytes)
        Approximate cache size: 0.00 Byte (0 blocks), need-reset=no
           Physical block size: 512.00 Byte (2^9 Bytes)

Probe time: 18'47"

-- end of output --

That's both sticks : NB the 2nd took  3 times  as long.

So both sticks are genuine, as I would expect from that store.

I plan to try 'gparted' next, instead of 'fdisk'
& will send the results when I get them.

BTW yes, we do have consumer protection laws in Ontario.
I once was able to exchange a defective dictionary at a bookstore :
one of the signatures (bindings) was missing & another was repeated,
so it was clearly unfit for its purpose & someone somewhere shd have checked.

Also, the sticks didn't feel warm after their exercises ;
'glance' is in Gentoo & might help show the details :
I'll have a look at it.

Thanks for all the interest : there's more yet (smile) !

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

[gentoo-user] Re: problem formatting new 256 GB USB stick : f3

[Grant Edwards]

On 2025-02-18, Philip Webb <purslow@ca.inter.net> wrote:

> So both sticks are genuine, as I would expect from that store.
>
> I plan to try 'gparted' next, instead of 'fdisk'
> & will send the results when I get them.

Have you tried just doing mkfs.ext4 [options] /dev/sdb?

You only want one filesystem, right?

Why bother with a partition table if you don't want to partition the
device?

--
Grant

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : f3

[Philip Webb]

250218 Grant Edwards wrote:
> On 2025-02-18, Philip Webb <purslow@ca.inter.net> wrote:
>> So both sticks are genuine, as I would expect from that store.
> Have you tried just doing mkfs.ext4 [options] /dev/sdb?
> You only want one filesystem, right?  Why bother with a partition table
> if you don't want to partition the device?

It had an M$ partition when I bought it.  I replaced that with  4  partitions
because using a 2/3.2 port took  > 10 h  to write a  256 GB filesystem & fail,
whereas it took only  2 h 45 m  to write a  64 GB  partition & fail.

FYI I bought  4  Kingston  128 GB  sticks in 2023 & had no problem with them.
At that time, the store didn't offer  256 GB  sticks.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : f3

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 1718 bytes --]

Am Tue, Feb 18, 2025 at 12:53:42AM -0500 schrieb Philip Webb:
> 250218 Grant Edwards wrote:
> > On 2025-02-18, Philip Webb <purslow@ca.inter.net> wrote:
> >> So both sticks are genuine, as I would expect from that store.
> > Have you tried just doing mkfs.ext4 [options] /dev/sdb?
> > You only want one filesystem, right?  Why bother with a partition table
> > if you don't want to partition the device?
> 
> It had an M$ partition when I bought it.  I replaced that with  4  partitions
> because using a 2/3.2 port took  > 10 h  to write a  256 GB filesystem & fail,
> whereas it took only  2 h 45 m  to write a  64 GB  partition & fail.
> 
> FYI I bought  4  Kingston  128 GB  sticks in 2023 & had no problem with them.
> At that time, the store didn't offer  256 GB  sticks.

BTW, in case you come into this position again that you need mobile storage. 
I recommend an external USB case with an NVMe SSD inside. This may not be as 
compact and not as cheap, but they are much much much faster and prolly 
longer-lasting than any USB “stick”, b/c their flash controllers are more 
sophisticated and the parts of higher fidelity. A not-too-cheap NVMe will do 
fine, as long as it’s TLC, not QLC flash. And when used behind USB, an 
on-board DRAM cache is beneficial if you deal with many small files.

A good metal case with 10 Gbps USB is around 20 bucks, and the older Kioxia 
Exceria G2 with 500 GB and DRAM can be had for 35 € on the Old Continent’s 
market. No idea about Canadian prices though.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Latin: the late revenge of the Romans to all Germans.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : alternatives

[Philip Webb]

250218 Frank Steinmetzger wrote:
> I recommend an external USB case with an NVMe SSD inside.
> This may not be as compact and not as cheap,
> but they are much much much faster and probably longer-lasting
> than any USB stick, because their flash controllers are more sophisticated
> and the parts of higher fidelity.  A not-too-cheap NVMe will do fine,
> as long as it’s TLC, not QLC flash.

Can you explain this -- TLC/QLC -- in more detail ?

> And when used behind USB, an on-board DRAM cache is beneficial
> if you deal with many small files.

Is that a different solution or part of the same device ?

> A good metal case with  10 Gb/s  USB is  c USD 20
> and the older Kioxia Exceria G2 with  500 GB and DRAM
> can be had for  35 €  on the Old Continent’s market.

That looks very interesting, if CC sells it or someone reliable via I/net.

I use a USB 3 stick with  32 GB  for daily back-ups
& USB 2 or 3's with  64 - 128 GB  for longer-term archives ;
smaller sticks are necessary to live-system ISOs.

USB sticks are very economical with physical space,
but if they're becoming less reliable at the manufacturing end,
it would make sense to try something which promises to be more reliable.

Does anyone else comments on this or other alternative solutions ?

> Latin: the late revenge of the Romans to all Germans.

To a native Anglophone, German looks more like someone's revenge (grin).

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

[gentoo-user] Re: problem formatting new 256 GB USB stick : alternatives

[Grant Edwards]

On 2025-02-18, Philip Webb <purslow@ca.inter.net> wrote:
> 250218 Frank Steinmetzger wrote:
>> I recommend an external USB case with an NVMe SSD inside.
>> This may not be as compact and not as cheap,
>> but they are much much much faster and probably longer-lasting
>> than any USB stick, because their flash controllers are more sophisticated
>> and the parts of higher fidelity.  A not-too-cheap NVMe will do fine,
>> as long as it’s TLC, not QLC flash.
>
> Can you explain this -- TLC/QLC -- in more detail ?

Tri-Level Cells,  Quad-Level Cells.

SLC (single-level cells)

Each "cell" consists of a capacitor (constructed as a FET with an
isolated-gate).  In the old days, the capacitor was either chared or
not: two levels.  One bit per cell.  Easy to make and reliable.  If
you want more bits, layout more cells. That means larger dies, which
means more money.

MLC (two-level cells)

OK, let's also do four different charge levels!  You get _two_ bits
per cell with the same area of silicon.  Harder to do and not quite as
reliable, so we'll add a few extra blocks of cells and make software
deal with blocks that don't work.

TLC (tri-level cells)

Hey, what if we do 8 different charge levels! Now you get 4 bits per
cell.  Even harder to do, even less reliable...


QLC (quad-level cells)

[I think you see where this is going]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : alternatives

[Michael]

[-- Attachment #1: Type: text/plain, Size: 2168 bytes --]

On Tuesday 18 February 2025 21:18:03 Greenwich Mean Time Grant Edwards wrote:
> On 2025-02-18, Philip Webb <purslow@ca.inter.net> wrote:
> > 250218 Frank Steinmetzger wrote:
> >> I recommend an external USB case with an NVMe SSD inside.
> >> This may not be as compact and not as cheap,
> >> but they are much much much faster and probably longer-lasting
> >> than any USB stick, because their flash controllers are more
> >> sophisticated
> >> and the parts of higher fidelity.  A not-too-cheap NVMe will do fine,
> >> as long as it’s TLC, not QLC flash.
> > 
> > Can you explain this -- TLC/QLC -- in more detail ?
> 
> Tri-Level Cells,  Quad-Level Cells.
> 
> SLC (single-level cells)
> 
> Each "cell" consists of a capacitor (constructed as a FET with an
> isolated-gate).  In the old days, the capacitor was either chared or
> not: two levels.  One bit per cell.  Easy to make and reliable.  If
> you want more bits, layout more cells. That means larger dies, which
> means more money.
> 
> MLC (two-level cells)
> 
> OK, let's also do four different charge levels!  You get _two_ bits
> per cell with the same area of silicon.  Harder to do and not quite as
> reliable, so we'll add a few extra blocks of cells and make software
> deal with blocks that don't work.
> 
> TLC (tri-level cells)
> 
> Hey, what if we do 8 different charge levels! Now you get 4 bits per
> cell.  Even harder to do, even less reliable...
> 
> 
> QLC (quad-level cells)
> 
> [I think you see where this is going]

Yep, SLC > MLC > TLC > QLC in terms of performance, reliability, longevity and 
price.  The phenomenal decrease in price per GB/TB over time is just that.  
Phenomenal.  For reasons Grant explained it comes with measured reduction in 
reliability and longevity.  Since they come with fast(er) DRAM cache which to 
some extent masks the underlying NAND performance, with other things being 
equal it makes sense to choose one with faster/larger DRAM.  I think most 
consumer grade cards these days tend to be TLC and the cheaper ones QLC.  QLC 
ought to outlast a conventional USB stick, but given a choice I'd opt for TLC.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : alternatives

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 1631 bytes --]

Am Tue, Feb 18, 2025 at 11:25:28PM +0000 schrieb Michael:

> > TLC (tri-level cells)
> > 
> > Hey, what if we do 8 different charge levels! Now you get 4 bits per
> > cell.  Even harder to do, even less reliable...
> > 
> > 
> > QLC (quad-level cells)
> > 
> > [I think you see where this is going]
> 
> Yep, SLC > MLC > TLC > QLC in terms of performance, reliability, longevity and 
> price.  The phenomenal decrease in price per GB/TB over time is just that.  
> Phenomenal.  For reasons Grant explained it comes with measured reduction in 
> reliability and longevity.

> Since they come with fast(er) DRAM cache which to 
> some extent masks the underlying NAND performance, with other things being 
> equal it makes sense to choose one with faster/larger DRAM.

Small correction: the DRAM is not for fast write performance. That is what 
SLC cache is for: a small(ish) part of the TLC or QLC flash is used in SLC 
mode, so the SSD can ingest data faster. In time of idleness, the SSD 
controller starts “swapping out” the content of the SLC cache into its slow 
TLC/QLC storage.

The DRAM’s single purpose is to hold the flash translation layer, i.e. the 
mapping of logical storage area to physical flash cell. Otherwise it would 
have to constantly be written during operation, which will make such 
operation slower, epecially when dealing with many small files.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Development aid is to give money from the poor of rich countries
to the rich of poor countries.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : alternatives

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 1581 bytes --]

Am Tue, Feb 18, 2025 at 02:10:35PM -0500 schrieb Philip Webb:

> 250218 Frank Steinmetzger wrote:
> > I recommend an external USB case with an NVMe SSD inside.
> > This may not be as compact and not as cheap,
> > but they are much much much faster and probably longer-lasting
> > than any USB stick, because their flash controllers are more sophisticated
> > and the parts of higher fidelity.  A not-too-cheap NVMe will do fine,
> > as long as it’s TLC, not QLC flash.
> 
> Can you explain this -- TLC/QLC -- in more detail ?

Grant provided a nice explanation.

> > A good metal case with  10 Gb/s  USB is  c USD 20
> > and the older Kioxia Exceria G2 with  500 GB and DRAM
> > can be had for  35 €  on the Old Continent’s market.
> 
> That looks very interesting, if CC sells it or someone reliable via I/net.

One thing I forgot to mention: AFAIK not all such enclosures support TRIM. 
If you want to get one, check the conroller chip and whether it supports 
TRIM. In the USB world, the relevant feature is called UAS/UASP (USB 
Attached SCSI).

> I use a USB 3 stick with  32 GB  for daily back-ups
> & USB 2 or 3's with  64 - 128 GB  for longer-term archives ;
> smaller sticks are necessary to live-system ISOs.

Yes, sticks a nice’n small. And I also have some quite old ones between 
8 and 32 Gigs with Ventoy on them for booting purposes.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Emacs is a great operating system, which only lacks a good editor.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : alternatives

[Michael]

[-- Attachment #1: Type: text/plain, Size: 941 bytes --]

On Wednesday 19 February 2025 12:20:33 Greenwich Mean Time Frank Steinmetzger 
wrote:
> Am Tue, Feb 18, 2025 at 02:10:35PM -0500 schrieb Philip Webb:
> > 250218 Frank Steinmetzger wrote:

> One thing I forgot to mention: AFAIK not all such enclosures support TRIM.
> If you want to get one, check the conroller chip and whether it supports
> TRIM. In the USB world, the relevant feature is called UAS/UASP (USB
> Attached SCSI).

I was about to mention this ... as Frank explained not all external drives are 
built equal.  TRIM is necessary if you want the performance of the drive to 
remain high as it gets (re)filled up and not suffer from 'write 
amplification'.  This functionality is provided by the onboard controller and 
has to be supported by the filesystem too.  It used to be the case FAT and 
exFAT did not make use of TRIM, or at least they did not when used with 
MSWindows, but I think these days all(?) filesystems use it.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[gentoo-user] Re: problem formatting new 256 GB USB stick : f3

[Grant Edwards]

On 2025-02-18, Philip Webb <purslow@ca.inter.net> wrote:
> 250218 Grant Edwards wrote:
>> On 2025-02-18, Philip Webb <purslow@ca.inter.net> wrote:
>>> So both sticks are genuine, as I would expect from that store.
>> Have you tried just doing mkfs.ext4 [options] /dev/sdb?
>> You only want one filesystem, right?  Why bother with a partition table
>> if you don't want to partition the device?
>
> It had an M$ partition when I bought it.  I replaced that with  4  partitions
> because using a 2/3.2 port took  > 10 h  to write a  256 GB filesystem & fail,
> whereas it took only  2 h 45 m  to write a  64 GB  partition & fail.

There is something _seriously_ wrong with the device.

  1. It shouldn't take anywhere near that wrong to create a
     filesystem: there just isn't that much data written during an
     'mkfs' operation.

  2. Writing even that small amount of data FAILED.

I can't understand why you want to continue using this device.

> FYI I bought 4 Kingston 128 GB sticks in 2023 & had no problem with
> them.  At that time, the store didn't offer 256 GB sticks.

--
Grant

[gentoo-user] Re: problem formatting new 256 GB USB stick : f3

[Grant Edwards]

On 2025-02-18, Grant Edwards <grant.b.edwards@gmail.com> wrote:
>
>> It had an M$ partition when I bought it.  I replaced that with 4
>> partitions because using a 2/3.2 port took > 10 h to write a 256 GB
>> filesystem & fail, whereas it took only 2 h 45 m to write a 64 GB
>> partition & fail.
>
> There is something _seriously_ wrong with the device.
>
>   1. It shouldn't take anywhere near that wrong to create a
>      filesystem: there just isn't that much data written during an
>      'mkfs' operation.

It also shouldn't take anywhere near that _long_ to create a
filesystem. :)

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : f3

[Michael]

[-- Attachment #1: Type: text/plain, Size: 4593 bytes --]

On Tuesday 18 February 2025 03:46:26 Greenwich Mean Time Philip Webb wrote:
> 250217 Michael wrote:
> > It is worth mentioning the sys-block/f3 package (Fight Flash Fraud),
> > which is in Portage and can test a USB flash disk to discover if it is
> > fake. Besides the slower f3write and f3read, the f3probe command
> > will only take a few minutes and confirm the available space.
> 
> Thanks ! -- 'f3write' + 'f3read' refused, as they expect a directory ;

The man page explains how to run these commands.  You need to provide the 
mountpoint directory for the device after you mount it, e.g.:

$ f3write /run/media/<username>/XXXX-XXXX

then,

$ f3read /run/media/<username>/XXXX-XXXX

You'd probably want to create one big partition for the whole device, or no 
partition at all, i.e. run something like this:

# mkfs.fat -F 32 -n New-USB1 /dev/sdb

The f3write/f3read commands will check the large files written by f3 can be 
accessed and read without any problem and flag up any sectors which contain 
corrupted data.


> 'f3probe' need it to be compiled with 'USE="extra"', but does work :
> 
> root:705 ~> f3probe /dev/sdb

The f3probe command needs to be run as root, after you unmount the device.  
From your prompt I expect you did this, but it is also advisable to run it 
destructively - any data on the sticks will be overwritten:

f3probe --destructive --time-ops /dev/sdb


> F3 probe 8.0
> Copyright (C) 2010 Digirati Internet LTDA.
> This is free software; see the source for copying conditions.
> 
> WARNING: Probing normally takes from a few seconds to 15 minutes, but
>          it can take longer. Please be patient.
> 
> Probe finished, recovering blocks... Done
> 
> Good news: The device `/dev/sdb' is the real thing
> 
> Device geometry:
>                  *Usable* size: 231.05 GB (484540416 blocks)
>                 Announced size: 231.05 GB (484540416 blocks)
>                         Module: 256.00 GB (2^38 Bytes)
>         Approximate cache size: 0.00 Byte (0 blocks), need-reset=no
>            Physical block size: 512.00 Byte (2^9 Bytes)
> 
> Probe time: 5'29"
> root:706 ~> f3probe /dev/sdb
> F3 probe 8.0
> Copyright (C) 2010 Digirati Internet LTDA.
> This is free software; see the source for copying conditions.
> 
> WARNING: Probing normally takes from a few seconds to 15 minutes, but
>          it can take longer. Please be patient.
> 
> Probe finished, recovering blocks... Done
> 
> Good news: The device `/dev/sdb' is the real thing
> 
> Device geometry:
>                  *Usable* size: 231.05 GB (484540416 blocks)
>                 Announced size: 231.05 GB (484540416 blocks)
>                         Module: 256.00 GB (2^38 Bytes)
>         Approximate cache size: 0.00 Byte (0 blocks), need-reset=no
>            Physical block size: 512.00 Byte (2^9 Bytes)
> 
> Probe time: 18'47"
> 
> -- end of output --
> 
> That's both sticks : NB the 2nd took  3 times  as long.

This is good news, it confirms neither of them are counterfeit units.

However, the 2nd stick appears to be defective.  It takes almost 3.5 times as 
long than the first stick and from what we know for no good reason.  This 
indicates the second stick has bad flash cells, a bad flash controller, or 
both.

I don't know if checking for bad blocks when you format these drives may help 
at all.  You'd expect the flash controller to manage defective NAND cells 
transparently to the OS by abstracting the hardware to a logical layer.  
However, if the controller is not sophisticated as would be the case in a more 
expensive SSD drive and keeps trying repeatedly to write into bad cells, it 
might help to ask the filesystem to manage the bad blocks and see what you 
get.

Personally, I wouldn't bother and return the bad stick as defective, asking 
for it to be replaced.


> So both sticks are genuine, as I would expect from that store.

In my experience stores are box shifters.  Goods In - Goods Out.  They try to 
buy from importers/wholesalers with whom they have some relationship based on 
price-reliability-convenience and who they hope also apply similar criteria in 
their relationships up the supply chain.  Any link in this chain can go wrong 
and you end up with bad goods.  I don't expect them to perform QA/QC beyond 
looking at the shipping labels and customs declarations.  The manufacturers 
may perform some actual quality checks on a sampling basis after the 
prototypes have been put together and the brand reps may audit the odd 
production run.  I would think the rigour of such checks is proportional to 
the value of the assembled items.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Philip Webb]

250218 Michael wrote:

-- details of using 'f3probe' snipped --
 
> This is good news, it confirms neither of them are counterfeit units.
> However, the 2nd stick appears to be defective.  It takes almost 3.5 times
> as long than the first stick and from what we know for no good reason.
> This indicates the second stick has bad flash cells,
> a bad flash controller, or both.  I would return the bad stick as defective,
> asking for it to be replaced.
> 
> In my experience stores are box shifters.  Goods In - Goods Out.
> They try to buy from wholesalers with whom they have some relationship
> based on price-reliability-convenience and who they hope
> also apply similar criteria in their relationships up the supply chain.
> Any link in this chain can go wrong and you end up with bad goods.
> I don't expect them to perform QA/QC
> beyond looking at the shipping labels and customs declarations.
> The manufacturers may perform some actual quality checks on a sampling basis
> after the prototypes have been put together
> and the brand reps may audit the odd production run.
> I would think the rigour of such checks
> is proportional to the value of the assembled items.

Yes, that's very much what one would expect of any successful retailer.
My experience with Canada Computers since 2000 --  25 yr  -- has been good :
they do try not to sell anything which cb unreliable.
However, as you say, all it takes is  1  broken link in a long chain.

I tried formatting one of the sticks using Gparted + Glances.
After  10 hr  while I was asleep, it had formatted  1  partition
& was clearly stalled on the 2nd : the Parted progress bar was moving to-fro,
Glances reported critically high CPU I/O wait of  c 25 %
& that it was writing small amounts of data to the partition every  30 sec .
I had to cancel Gparted from the root command line
& then tested the 1st partition, which allowed me to edit a file,
but was very slow at everything.

So yes, at least  1  of the sticks is unusable & probably both.
I can take/mail them back to the store & ask them to test them with Linux
& refund my CAD if they confirm they're defective.
I would expect this store to be honorable re it
& to be pleased to have someone report the problem
st they can report back to their supplier & avoid future trouble.

Frank had a useful suggestion re alternative devices,
which I will reply to separately.  Your own comments thereon wb useful.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

On Tuesday 18 February 2025 18:54:07 Greenwich Mean Time Philip Webb wrote:

> So yes, at least  1  of the sticks is unusable & probably both.
> I can take/mail them back to the store & ask them to test them with Linux
> & refund my CAD if they confirm they're defective.

I would refrain from stating anything which could well be outside their sphere 
of knowledge or understanding.  Hence I would not mention Linux, ext2, 
gparted, or anything exotic for the average MSWindows user, beyond:

"I just cannot format these sticks - they took many, many hours and eventually 
failed.  They are not like the previous USB sticks I bought FROM YOUR SHOP.  
Can I please replace them for something more reliable AND FASTER.  If need be 
at a higher price point."


> I would expect this store to be honorable re it
> & to be pleased to have someone report the problem
> st they can report back to their supplier & avoid future trouble.
> 
> Frank had a useful suggestion re alternative devices,
> which I will reply to separately.  Your own comments thereon wb useful.

For reliable NAND flash storage on a modern PC which can make use of the 
higher speeds, I wholeheartedly agree the M.2 small form factor SSD drive 
within a USB enclosure must be a consideration.  Or one of the external SATA 
SSDs which are physically bigger, with a USB cable. 

There are also USB enclosures (caddies), in which you can buy and install a 
suitable M.2 or 2.5" SATA SSD of your choice; e.g.:

https://plugable.com/products/usbc-nvme/

The SSDs come with a fast DRAM cache which accelerates transfers of smaller 
files, so the difference in speed from your flaky USB sticks will be very 
noticeable.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Grant Edwards]

On 2025-02-18, Michael <confabulate@kintzios.com> wrote:
> On Tuesday 18 February 2025 18:54:07 Greenwich Mean Time Philip Webb wrote:
>
>> So yes, at least 1 of the sticks is unusable & probably both.  I
>> can take/mail them back to the store & ask them to test them with
>> Linux & refund my CAD if they confirm they're defective.
>
> I would refrain from stating anything which could well be outside
> their sphere of knowledge or understanding.  Hence I would not
> mention Linux, ext2, gparted, or anything exotic for the average
> MSWindows user, beyond:

Definitely that. The second they hear "Linux", they're going to say
"Those aren't supported on Linux" and turn their backs. If you push
the issue, they're going to claim the waranty is void because you were
using Linux. Well, that's what would happen in US. Maybe retailers are
nicer in Canada?

> "I just cannot format these sticks - they took many, many hours and
> eventually failed.  They are not like the previous USB sticks I
> bought FROM YOUR SHOP.  Can I please replace them for something more
> reliable AND FASTER.  If need be at a higher price point."

If they start asking questions, just say: "I tried to format them on
my PC. It ran for 10 hours, then failed.".  It doesn't matter what
they ask apart from "cash refund, replacement or credit?", that's your
answer.

I've bought a number of Toshiba and Samsung internal and external SSDs
(M.2 and USB3), and they have absolutely solid.  If possible by flash
products from manufacturers that actually make flash chips. You know
they're not putting the floor sweepings in their own products.

--
Grant

[gentoo-user] alternatives to USB sticks for reliable archiving

[Philip Webb]

250219 Grant Edwards wrote:
> On 2025-02-18, Michael <confabulate@kintzios.com> wrote:
>> I would refrain from stating anything which could well be
>> outside their sphere of knowledge or understanding.
>> I would not mention Linux, ext2, gparted
>> or anything exotic for the average MSWindows user.
>> "I just cannot format these sticks - they took many, many hours
>> and eventually failed.  They are not like the previous USB sticks
>> I bought FROM YOUR SHOP.  Can I please replace them
>> for something more reliable AND FASTER, if need be at a higher price".
> Definitely that.  The second they hear "Linux", they're going to say
> "Those aren't supported on Linux" and turn their backs.
> If you push the issue, they're going to claim the waranty is void
> because you were using Linux.

Then they should definitely advertise the fact that they work only for M$ ;
USB sticks sb reformatable at the customer's wish :
if these aren't, the store should make that clear to the purchaser.
CC is the kind of store you go to if you build your own machines,
so I wb surprised if that was their response ; it's more likely to be :
"They are outside the in-store warranty. You can send them back to the maker".

> Well, that's what would happen in US. Maybe retailers are nicer in Canada ?

Yes, we're all nicer in Canada : that's why the US Prez doesn't like us (grin)

> If they start asking questions, just say: "I tried to format them on
> my PC. It ran for 10 hours, then failed".  It doesn't matter what they ask
> apart from "cash refund, replacement or credit?", that's your answer.

No, you're both quite wrong. "Both sticks came already formatted as 'msdos',
so why were you trying to reformat them, Mr Customer ?
As you should know as a Linux user, Linux systems can RWX 'msdos' files".
To which there is no clear-cut reply.

> I've bought a number of Toshiba and Samsung internal and external SSDs
> (M.2 and USB3) and they have been absolutely solid.  If possible,
> buy flash products from manufacturers that actually make flash chips.
> You know they're not putting the floor sweepings in their own products.

Thanks to both + Frank : I'll definitely look into this.

I have plenty of older but reliable sticks,
but if I can't rely on new ones in future,
I'll need an alternative for long-term system archives.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatcadotinterdotnet

[gentoo-user] Re: alternatives to USB sticks for reliable archiving

[Grant Edwards]

On 2025-02-19, Philip Webb <purslow@ca.inter.net> wrote:

>> If they start asking questions, just say: "I tried to format them on
>> my PC. It ran for 10 hours, then failed".  It doesn't matter what they ask
>> apart from "cash refund, replacement or credit?", that's your answer.
>
> No, you're both quite wrong. "Both sticks came already formatted as
> 'msdos', so why were you trying to reformat them, Mr Customer?

"I tried to format them on my PC. It ran for 10 hours, then failed."

> As you should know as a Linux user, Linux systems can RWX 'msdos'
> files".  To which there is no clear-cut reply.

You don't say you're a Linux user.  You say

"I tried to format them on my PC. It ran for 10 hours, then failed."

In a pinch, you can say.

"All of the other ones I bought here formatted with no problems on my PC."

Say "PC" a lot.  In their mind "PC" == "Windows".  It will never even
occur to them that you're running Linux.

--
Grant

Re: [gentoo-user] alternatives to USB sticks for reliable archiving

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1854 bytes --]

On Wednesday 19 February 2025 01:12:34 Greenwich Mean Time Philip Webb wrote:
> 250219 Grant Edwards wrote:

> > If they start asking questions, just say: "I tried to format them on
> > my PC. It ran for 10 hours, then failed".  It doesn't matter what they ask
> > apart from "cash refund, replacement or credit?", that's your answer.
> 
> No, you're both quite wrong. "Both sticks came already formatted as 'msdos',
> so why were you trying to reformat them, Mr Customer ?

I smile at the mental gymnastics we have to go through to avoid being advised 
to restart our PC ...  :p

The answer to the hypothetical question the cocky shop assistant would ask is 
to say you HAD TO format the stick because when you attempted to copy a couple 
of 2GB file(s) it took many hours and it failed.

To confirm this would be the case you can just create a single /dev/sdb1 
partition the size of the whole drive, format it[1], before you try to copy 5 
or 6 large files, like e.g. a few full ISOs.   Given what you've observed so 
far I expect one or both of the sticks will fail.

Either way, Kingston will not ask such questions if you return them and should 
readily replace them.  Their DataTraveler sticks come with a 5 year warranty 
and free technical support.  In addition their data sheet states they work 
with MSWindows, MacOS, Linux and ChromeOS.  There are no caveats to avoid 
partitioning and formatting them with any other fs type than what they arrived 
with.

[1] Until recently MSWindows would not allow formatting a partition larger 
than 32GB with FAT, but they increased this to 2TB on a Windows 11 update.  
Therefore these sticks would probably have arrived with exFAT or NTFS 
filesystem format on them.

BTW, the interwebs have posts by a number of disgruntled customers complaining 
of glacial performance, so it seems you're not alone.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[eric]

On 2/18/25 16:13, Michael wrote:
> For reliable NAND flash storage on a modern PC which can make use of the
> higher speeds, I wholeheartedly agree the M.2 small form factor SSD drive
> within a USB enclosure must be a consideration.  Or one of the external SATA
> SSDs which are physically bigger, with a USB cable.

I have been using a Samsung portable SSD with USB 3 connectivity for 
quite some time. I used to work in an area with very limited internet 
for weeks at a time and used the disk to put some media to read, watch, 
or listen on it at home that I could take to my work location to use 
during my time off. The transfer speed after upgrading from a standard 
USB drive to the SSD was amazing and I never looked back. Of course, 
there is a price increase for the increased speed.

Eric

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Dale]

Michael wrote:
> For reliable NAND flash storage on a modern PC which can make use of the 
> higher speeds, I wholeheartedly agree the M.2 small form factor SSD drive 
> within a USB enclosure must be a consideration.  Or one of the external SATA 
> SSDs which are physically bigger, with a USB cable. 
>
> There are also USB enclosures (caddies), in which you can buy and install a 
> suitable M.2 or 2.5" SATA SSD of your choice; e.g.:
>
> https://plugable.com/products/usbc-nvme/
>
> The SSDs come with a fast DRAM cache which accelerates transfers of smaller 
> files, so the difference in speed from your flaky USB sticks will be very 
> noticeable.


I use external USB sticks a lot for critical backup files, world, /etc
and most important, /root where some of my so called scripts live. 
Reading this thread made me question even more the dependability of USB
sticks.  I've read elsewhere that even some highly respected brands are
heading down the road to being iffy.  So, I just bought a enclosure and
a m.2 stick.  This is what I got. 

https://www.amazon.com/dp/B0CYLDM23M

https://www.amazon.com/dp/B0DQSX3Z76

I'll likely have one partition that is not encrypted and others that
are.  I hope those two work together.  From what I understand of reading
in this thread, this is a more dependable option.  These SSDs have come
a long ways on the reliability front compared to several years ago.  The
$70 price tag is more than a USB stick BUT as it is, I have the same
info across multiple USB sticks just to hope that at least one of them
won't fail.  Added together, those aren't cheap either.  As most know, I
have a couple fire safes.  I have one that is supposed to be media
rated.  Still, I put really important things in a fireproof/resistant
bag.  It gives them a little more protection.  I haven't figured out how
to do that with that hard drive enclosure that holds five hard drives
yet tho.  I'm hoping this will be a much more reliable method. 

This is why I read threads even if I can't help or anything.  I learn
something.  I just wonder if I can encrypt stuff on there.  I bet it
works like anything else. 

Dale

:-)  :-) 

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 3301 bytes --]

Am Thu, Feb 20, 2025 at 12:18:30PM -0600 schrieb Dale:
> Michael wrote:

> I use external USB sticks a lot for critical backup files, world, /etc
> and most important, /root where some of my so called scripts live. 
> Reading this thread made me question even more the dependability of USB
> sticks.  I've read elsewhere that even some highly respected brands are
> heading down the road to being iffy.  So, I just bought a enclosure and
> a m.2 stick.  This is what I got. 
> 
> https://www.amazon.com/dp/B0CYLDM23M
> 
> https://www.amazon.com/dp/B0DQSX3Z76


You should have announced it beforehand, so we could have given you input as 
usual. :) I’m sure it will work just fine as temporary storage, but don’t 
use it as singular backup device without, well, backup. Because I feel the 
urge to dampen your mood by saying that the E100 is a very very cheap model, 
if not the cheapest of all from any of the big named brands. It’s probably 
still better than sticks, I guess.

The E100 just came out, so there are no long-term experiences available yet.
There has been a news item on my German hardware site on its recent release: 
https://www.computerbase.de/news/storage/heimlich-eingefuehrt-ssd-serie-crucial-e100-erreicht-den-markt-im-tbw-tiefflug.91090/

Google-translated to English:
https://www-computerbase-de.translate.goog/news/storage/heimlich-eingefuehrt-ssd-serie-crucial-e100-erreicht-den-markt-im-tbw-tiefflug.91090/?_x_tr_sl=de&_x_tr_tl=en&_x_tr_hl=de&_x_tr_pto=wapp

The ghist of the article:
- It’s so low-profile there was no press announcement by the manufacturer.
- According to the article, it has a new record low for guaranteed lifetime 
  writes (TBW, terabytes written). The article has a nice table of 
  comparison with many other SSDs. The next best has more than twice the 
  amount.
- The (slighty harsh) comments speak of leftover recycling. The users are 
  hard-core gamers though, so the best is just good enough for them.

https://www.techpowerup.com/ssd-specs/crucial-e100-480-gb.d2306 mentions TLC 
and no DRAM, though the computerbase article speaks of QLC (which is more 
likely, especially given the TWB value). So don’t expect a speed miracle, 
especially behind USB.

Another thing to consider: don’t put it into the safe for a year without 
powering it up. As was explained in a previous mail, QLC uses sixteen 
different levels of charge inside one single flash cell. The chance of a bit 
flip increases the longer the SSD is powerless and charge slowly (very 
slowly) dissipates. It’s hard to find exact numbers, and it’s more of a 
statistical question. Could be a research topic for a slow Sunday. ;-) 
Also, don’t you live in a hot area?

I don’t know if an SSD does a re-charge regularly by itself, or whether you 
need to actually read out the cells for the controller to notice any issues. 
This may be another good research topic. You could simply tar the entire 
mounted SSD to /dev/null to read the entire payload once (without reading 
the unused areas to save time):
tar cf /dev/null /path/to/filesystem/

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

A circle is a round square.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Grant Edwards]

On 2025-02-20, Frank Steinmetzger <Warp_7@gmx.de> wrote:

> Another thing to consider: don’t put it into the safe for a year without 
> powering it up. As was explained in a previous mail, QLC uses sixteen 
> different levels of charge inside one single flash cell. The chance of a bit 
> flip increases the longer the SSD is powerless and charge slowly (very 
> slowly) dissipates. It’s hard to find exact numbers, and it’s more of a 
> statistical question. Could be a research topic for a slow Sunday. ;-) 
> Also, don’t you live in a hot area?
>
> I don’t know if an SSD does a re-charge regularly by itself, or whether you 
> need to actually read out the cells for the controller to notice any issues. 
> This may be another good research topic.

According to some sources I've read, (at least some) flash controllers
do automatic refresh cycles in the background. I've no clue how long a
complete device refresh cycle would take for any particular device, or
how long between refresh cycles you would want to tolerate.

--
Grant

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Dale]

Grant Edwards wrote:
> On 2025-02-20, Frank Steinmetzger <Warp_7@gmx.de> wrote:
>
>> Another thing to consider: don’t put it into the safe for a year without 
>> powering it up. As was explained in a previous mail, QLC uses sixteen 
>> different levels of charge inside one single flash cell. The chance of a bit 
>> flip increases the longer the SSD is powerless and charge slowly (very 
>> slowly) dissipates. It’s hard to find exact numbers, and it’s more of a 
>> statistical question. Could be a research topic for a slow Sunday. ;-) 
>> Also, don’t you live in a hot area?
>>
>> I don’t know if an SSD does a re-charge regularly by itself, or whether you 
>> need to actually read out the cells for the controller to notice any issues. 
>> This may be another good research topic.
> According to some sources I've read, (at least some) flash controllers
> do automatic refresh cycles in the background. I've no clue how long a
> complete device refresh cycle would take for any particular device, or
> how long between refresh cycles you would want to tolerate.
>
> --
> Grant

Well, I may swap it out with a better brand later.  I tend to update
those backups every month or so.  They don't change a lot.  Should have
stuck with Samsung or whatever, even tho it costs twice as much.  I
could use the Crusial just to move data from one system to another
still.  Should be good for that.

Dale

:-)  :-) 

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 1566 bytes --]

Am Thu, Feb 20, 2025 at 11:40:48PM +0100 schrieb Frank Steinmetzger:

> Another thing to consider: don’t put it into the safe for a year without 
> powering it up. As was explained in a previous mail, QLC uses sixteen 
> different levels of charge inside one single flash cell. The chance of a bit 
> flip increases the longer the SSD is powerless and charge slowly (very 
> slowly) dissipates. It’s hard to find exact numbers, and it’s more of a 
> statistical question. Could be a research topic for a slow Sunday. ;-) 
> Also, don’t you live in a hot area?

I knew I’ve seen this data in the past, but couldn’t remeber where and in 
what context. I just stumbled upon the relevant info again.

If you search for "jedec temperature and data retention" you find this PDF:
https://www.jedec.org/sites/default/files/Alvin_Cox%20%5BCompatibility%20Mode%5D_0.pdf

It contains establised standard values for flash durability. On slide 28 it 
says:
Client-class SSDs should retain their data w/o power for 1 year if used
8 hours/day at 40 °C and kept at or below 30 °C when off.
There is also a table for other temperatures; the time is cut in half for
5 degrees more.

Mind you, that PDF is 15 years old, TLS had just been released to the public 
one year earlier according to https://en.wikipedia.org/wiki/Multi-level_cell#Triple-level_cell

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Some are so convinced, they don’t even know anymore of what.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Dale]

[-- Attachment #1: Type: text/plain, Size: 2441 bytes --]

Frank Steinmetzger wrote:
> Am Thu, Feb 20, 2025 at 11:40:48PM +0100 schrieb Frank Steinmetzger:
>
>> Another thing to consider: don’t put it into the safe for a year without 
>> powering it up. As was explained in a previous mail, QLC uses sixteen 
>> different levels of charge inside one single flash cell. The chance of a bit 
>> flip increases the longer the SSD is powerless and charge slowly (very 
>> slowly) dissipates. It’s hard to find exact numbers, and it’s more of a 
>> statistical question. Could be a research topic for a slow Sunday. ;-) 
>> Also, don’t you live in a hot area?
> I knew I’ve seen this data in the past, but couldn’t remeber where and in 
> what context. I just stumbled upon the relevant info again.
>
> If you search for "jedec temperature and data retention" you find this PDF:
> https://www.jedec.org/sites/default/files/Alvin_Cox%20%5BCompatibility%20Mode%5D_0.pdf
>
> It contains establised standard values for flash durability. On slide 28 it 
> says:
> Client-class SSDs should retain their data w/o power for 1 year if used
> 8 hours/day at 40 °C and kept at or below 30 °C when off.
> There is also a table for other temperatures; the time is cut in half for
> 5 degrees more.
>
> Mind you, that PDF is 15 years old, TLS had just been released to the public 
> one year earlier according to https://en.wikipedia.org/wiki/Multi-level_cell#Triple-level_cell
>
> -- Grüße | Greetings | Salut | Qapla’ Please do not share anything
> from, with or about me on any social network. Some are so convinced,
> they don’t even know anymore of what.


That is confusing.  Data should be OK for one year without power but
only if powered 8 hours per day.  Me scratches my head.  Maybe it means
should be OK for a year without power and not sure what to think about
rest. 

I suspect that things have changed a lot in that time frame despite the
confusion.  They may have improved things significantly since then. 
Still, if the one year part, less the other part, is accurate then but
better now, then my powering up even once a month should be OK.  I only
need to worry about if something happens to me and no one touches it for
many months, a year or longer. 

I think some maker somewhere needs to repeat these tests and see how
things stand now.  Then again, maybe the tech is changing so fast, they
can't keep up. 

Time to update my backups again.  :/ 

Dale

:-)  :-) 

[-- Attachment #2: Type: text/html, Size: 3592 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Michael]

[-- Attachment #1: Type: text/plain, Size: 4786 bytes --]

On Saturday, 8 March 2025 22:48:29 Greenwich Mean Time Dale wrote:
> Frank Steinmetzger wrote:
> > Am Thu, Feb 20, 2025 at 11:40:48PM +0100 schrieb Frank Steinmetzger:
> >> Another thing to consider: don’t put it into the safe for a year without
> >> powering it up. As was explained in a previous mail, QLC uses sixteen
> >> different levels of charge inside one single flash cell. The chance of a
> >> bit flip increases the longer the SSD is powerless and charge slowly
> >> (very slowly) dissipates. It’s hard to find exact numbers, and it’s more
> >> of a statistical question. Could be a research topic for a slow Sunday.
> >> ;-) Also, don’t you live in a hot area?
> > 
> > I knew I’ve seen this data in the past, but couldn’t remeber where and in
> > what context. I just stumbled upon the relevant info again.
> > 
> > If you search for "jedec temperature and data retention" you find this
> > PDF:
> > https://www.jedec.org/sites/default/files/Alvin_Cox%20%5BCompatibility%20M
> > ode%5D_0.pdf
> > 
> > It contains establised standard values for flash durability. On slide 28
> > it
> > says:
> > Client-class SSDs should retain their data w/o power for 1 year if used
> > 8 hours/day at 40 °C and kept at or below 30 °C when off.
> > There is also a table for other temperatures; the time is cut in half for
> > 5 degrees more.
> > 
> > Mind you, that PDF is 15 years old, TLS had just been released to the
> > public one year earlier according to
> > https://en.wikipedia.org/wiki/Multi-level_cell#Triple-level_cell
> > 
> > -- Grüße | Greetings | Salut | Qapla’ Please do not share anything
> > from, with or about me on any social network. Some are so convinced,
> > they don’t even know anymore of what.
> 
> That is confusing.  Data should be OK for one year without power but
> only if powered 8 hours per day.  Me scratches my head.  Maybe it means
> should be OK for a year without power and not sure what to think about
> rest. 

As I understand it these are two different performance requirements, which 
were stipulated in this proposed standard back in 2010.

The first, "Active Use (power on)", describes the level of performance on a 
typical client workload and operating environment, 8hrs/day at 40°C.  I don't 
know if this is controller temperature, NAND flash cell temperature, device 
sensor temperature, PC case temperature, or room temperature.  The actual 
standard document would explain this, as well as the method for measuring it.  
Meanwhile, I note my NVMe SSD's sensor tells me the drive is idling at 45.8°C 
as I write this.  o_O

The second, "Retention Use (power off)", sets a limit of 1 year @30°C.

The FFR and UBER thresholds apply to *both* of the above.


> I suspect that things have changed a lot in that time frame despite the
> confusion.  They may have improved things significantly since then. 

Hmm ... I wouldn't take this to the bank.  Standards are typically formulated 
following consultation with industry to establish the art of the possible, but 
without cutting too much slack on manufacturers so as to promote innovation 
and improvement.  The manufacturers seek to minimise overall costs, while 
keeping their head above the thresholds stated in the standard.  So there's a 
tension right there, which hopefully delivers innovation and ideally lowers 
costs over time - but it does not necessarily improve performance.  Check how 
quality of USB flash drives has gone south over the years, following the 
trajectory of their price.

The move from SLC to QLC has been effective in reducing costs, but at some 
point the limits of physics will constrain performance.  It would be 
interesting to dig out the latest JEDEC standard and compare thresholds to see 
how they may have improved with more modern manufacturing methods.


> Still, if the one year part, less the other part, is accurate then but
> better now, then my powering up even once a month should be OK.  I only
> need to worry about if something happens to me and no one touches it for
> many months, a year or longer. 

Given the temperature of the device would probably be below 30°C for at least 
half of the year in day time and most nights, I would think your data will be 
fine.  :-)

I have a 11 year old Toshiba OCZ SSD, which gave me a scare a week ago.  It 
reported an I/O error, dmesg showed it being inaccessible - I can't remember 
the exact error.  I thought it was game up with this drive, but surprisingly 
following my reseating the SATA cable and a reboot later it has worked without 
further complaints.  With reports of OCZ drives failing at an alarming rate 
more than a decade ago, I am happy it has lasted this long.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Dale]

[-- Attachment #1: Type: text/plain, Size: 5816 bytes --]

Michael wrote:
> On Saturday, 8 March 2025 22:48:29 Greenwich Mean Time Dale wrote:
>> Frank Steinmetzger wrote:
>>> Am Thu, Feb 20, 2025 at 11:40:48PM +0100 schrieb Frank Steinmetzger:
>>>> Another thing to consider: don’t put it into the safe for a year without
>>>> powering it up. As was explained in a previous mail, QLC uses sixteen
>>>> different levels of charge inside one single flash cell. The chance of a
>>>> bit flip increases the longer the SSD is powerless and charge slowly
>>>> (very slowly) dissipates. It’s hard to find exact numbers, and it’s more
>>>> of a statistical question. Could be a research topic for a slow Sunday.
>>>> ;-) Also, don’t you live in a hot area?
>>> I knew I’ve seen this data in the past, but couldn’t remeber where and in
>>> what context. I just stumbled upon the relevant info again.
>>>
>>> If you search for "jedec temperature and data retention" you find this
>>> PDF:
>>> https://www.jedec.org/sites/default/files/Alvin_Cox%20%5BCompatibility%20M
>>> ode%5D_0.pdf
>>>
>>> It contains establised standard values for flash durability. On slide 28
>>> it
>>> says:
>>> Client-class SSDs should retain their data w/o power for 1 year if used
>>> 8 hours/day at 40 °C and kept at or below 30 °C when off.
>>> There is also a table for other temperatures; the time is cut in half for
>>> 5 degrees more.
>>>
>>> Mind you, that PDF is 15 years old, TLS had just been released to the
>>> public one year earlier according to
>>> https://en.wikipedia.org/wiki/Multi-level_cell#Triple-level_cell
>>>
>>> -- Grüße | Greetings | Salut | Qapla’ Please do not share anything
>>> from, with or about me on any social network. Some are so convinced,
>>> they don’t even know anymore of what.
>> That is confusing.  Data should be OK for one year without power but
>> only if powered 8 hours per day.  Me scratches my head.  Maybe it means
>> should be OK for a year without power and not sure what to think about
>> rest. 
> As I understand it these are two different performance requirements, which 
> were stipulated in this proposed standard back in 2010.
>
> The first, "Active Use (power on)", describes the level of performance on a 
> typical client workload and operating environment, 8hrs/day at 40°C.  I don't 
> know if this is controller temperature, NAND flash cell temperature, device 
> sensor temperature, PC case temperature, or room temperature.  The actual 
> standard document would explain this, as well as the method for measuring it.  
> Meanwhile, I note my NVMe SSD's sensor tells me the drive is idling at 45.8°C 
> as I write this.  o_O
>
> The second, "Retention Use (power off)", sets a limit of 1 year @30°C.
>
> The FFR and UBER thresholds apply to *both* of the above.
>

They might should have explained both as two separate things instead of
both in almost one sentence.  The way it sounded to me, it was about the
same thing.


>> I suspect that things have changed a lot in that time frame despite the
>> confusion.  They may have improved things significantly since then. 
> Hmm ... I wouldn't take this to the bank.  Standards are typically formulated 
> following consultation with industry to establish the art of the possible, but 
> without cutting too much slack on manufacturers so as to promote innovation 
> and improvement.  The manufacturers seek to minimise overall costs, while 
> keeping their head above the thresholds stated in the standard.  So there's a 
> tension right there, which hopefully delivers innovation and ideally lowers 
> costs over time - but it does not necessarily improve performance.  Check how 
> quality of USB flash drives has gone south over the years, following the 
> trajectory of their price.
>
> The move from SLC to QLC has been effective in reducing costs, but at some 
> point the limits of physics will constrain performance.  It would be 
> interesting to dig out the latest JEDEC standard and compare thresholds to see 
> how they may have improved with more modern manufacturing methods.
>

I suspect SSD for m.2, 2.5" and such are still advancing as far as tech
goes.  USB stuff is likely falling off, except maybe for the really
large USB sticks and some major makers who still value quality. 


>> Still, if the one year part, less the other part, is accurate then but
>> better now, then my powering up even once a month should be OK.  I only
>> need to worry about if something happens to me and no one touches it for
>> many months, a year or longer. 
> Given the temperature of the device would probably be below 30°C for at least 
> half of the year in day time and most nights, I would think your data will be 
> fine.  :-)
>
> I have a 11 year old Toshiba OCZ SSD, which gave me a scare a week ago.  It 
> reported an I/O error, dmesg showed it being inaccessible - I can't remember 
> the exact error.  I thought it was game up with this drive, but surprisingly 
> following my reseating the SATA cable and a reboot later it has worked without 
> further complaints.  With reports of OCZ drives failing at an alarming rate 
> more than a decade ago, I am happy it has lasted this long.


I suspect it is safer than on a USB.  I believe that the old spinning
rust is likely the most durable long term storage without powering up
during storage.  I once hooked up a bunch of old IDE drives that hadn't
had power to them in years.  I went poking around and all the data was
still there.  It had some old videos, pictures, text files and other
stuff.  They all appeared fine to me.  Those drives sat in a out
building with no climate control at all.  Some were sitting on a shelf
bare, no static bag or anything.  They stored just fine.  Heck, I was
worried about the circuit boards more than anything.  Still, they worked. 

Dale

:-)  :-) 

[-- Attachment #2: Type: text/html, Size: 8299 bytes --]

Re: [gentoo-user] Re: problem formatting new 256 GB USB stick : glances at gparted

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 1322 bytes --]

Am Sun, Mar 09, 2025 at 10:32:37PM -0500 schrieb Dale:

> I suspect it is safer than on a USB.  I believe that the old spinning
> rust is likely the most durable long term storage without powering up
> during storage.  I once hooked up a bunch of old IDE drives that hadn't
> had power to them in years.  I went poking around and all the data was
> still there.  It had some old videos, pictures, text files and other
> stuff.  They all appeared fine to me.  Those drives sat in a out
> building with no climate control at all.  Some were sitting on a shelf
> bare, no static bag or anything.  They stored just fine.  Heck, I was
> worried about the circuit boards more than anything.  Still, they worked. 

HDD certainly have good long-term data retention properties in their 
magnetic platters. Their weakness is the mechanics. Especially with very old 
disks that haven’t run in ages, you might get stuck bearings because the 
lubricant lost its viscosity. Or on newer disks the helium diffuses out. 
AFAIK, helium drives should still work without it, but at reduced 
performance and they’ll run hotter.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

What is the Egyptian work for cowshed? – Moo-barrack.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] problem formatting new 256 GB USB stick

[Stroller]

> On 15 Feb 2025, at 07:41, Philip Webb <purslow@ca.inter.net> wrote:
> ...
> Has anyone else encountered this ?  Does anyone have suggestions ?

Create multiple large files on your desktop PC. You could use movie files if you have a small number of large enough files handy. I would prefer 5 files of about 45GB, and I would create them using `for i in {1..5} ; do dd if=/dev/random of=random$i ; done` and a file size specifier to dd.

md5sum the files, copy them to the drive and md5sum the copies. Do the md5sums match up?

I think there's a good chance they won't.

Re: [gentoo-user] problem formatting new 256 GB USB stick

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 2148 bytes --]

Am Mon, Feb 17, 2025 at 04:38:22PM +0000 schrieb Stroller:
> 
> 
> > On 15 Feb 2025, at 07:41, Philip Webb <purslow@ca.inter.net> wrote:
> > ...
> > Has anyone else encountered this ?  Does anyone have suggestions ?
> 
> Create multiple large files on your desktop PC. You could use movie files if you have a small number of large enough files handy. I would prefer 5 files of about 45GB, and I would create them using `for i in {1..5} ; do dd if=/dev/random of=random$i ; done` and a file size specifier to dd.
> 
> md5sum the files, copy them to the drive and md5sum the copies. Do the md5sums match up?

Methinks it should be enough to create one file of, say, 1 GB and copy that 
over with a new name each time until the drive is full. All files should 
return the same md5. This will also save you from writing dozens of GB to 
your main storage because you need its md5 beforehand (unless you have a 
big enough ramdisk).

NUM=1
while cp -v /path/to/source /path/to/stick/file$NUM; do NUM=$((NUM + 1))

When the drive is full, cp fails and the loop should end.

Or use a flash test program as was mentioned in a recent other post. Either 
way, going from the discussion so far, I would not trust those sticks until 
fully written to and read back. There are basically three tiers of Flash 
memory quality: the best goes into SSDs, the second-best goes into Memory 
cards or emmc (IIRC) and the “garbage” goes into USB sticks.

The computer store might be OK, but they have no control over what they get 
sent. There was a recrent scandal here in Germany about allegedly new 
Seagate HDDs being actually used drives from Chia farms whose SMART values 
have been reset. This only came to light because Seagate uses another set of 
internal diagnostic data that was not reset and showed several thousand 
hours of power-on-time. Several big retailers were affected because the 
problem was with the distributor.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

There are those and those.  But there are more of those than those.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]