Booting from NVMe USB enclosure

[Rapha_]

Looks like

1. writing on the system SSD NTFS is much slower than expected

2. reading from an NTFS partition on the USB is much slower than reading from the adjacent ext4 partition

Your Ntfs disk is fragmented ?

Are these tests valid ?

[rych]

Your Ntfs disk is fragmented ?

Oh, haven't heard that term for many years. I've goodled and found some strong opinions against it: Should I defrag my SSD, The short answer is no. The long answer is absolutely no. ; No, you should never defrag an SSD drive, only the old type HDD's need to be defragmented. An SSD uses a different technology named TRIM and that is managed automatically and so on.

Speaking of TRIM: ntfs-3g supports periodic TRIM, and ntfs3 -- continious, see https://wiki.archlinux.org/title/Solid_state_drive. But I've read it depends on the presence of the "discard" kernel parameter, and an enabled "fstrim.timer". I'm going to investigate this next.

Are these tests valid ?

-- I'm not sure what you mean. Or, I don't know. What other ways to run speed tests under Linux? But the very reason I started testing speeds is that NTFS (via ntfs-3g at least, ntfs3 to be investigated next) feels slow under Linux.

[Rapha_]

Pros:
1. Very fast, much faster than ntfs-3g.
2. Prevents fragmentation when writing files, whereas ntfs-3g generates a lot of fragmentation.
3. Low in CPU and RAM usage.

Cons:
1. NTFS3 has not been tested for stability, reliability and security. NTFS-3G has been tested for stability and safety.

NTFS3 Kernel Module (Post by babam #79250)



The results of these tests do not correspond at all to the manufacturer's data for my hard drive.
But maybe my disc is getting tired and becoming slower

[rych]

Rapha_, In that 3 years old post babam was retelling an even older story from the HDD era I guess. The world started seriously moving away from HDDs some 7-8 years ago, and SSDs apparently don't require defragmentation, see the links above. SSD controllers write all over the place by design to even out the wear or something.

As for dd benchmarking, I just copied those commands from other relevant forums -- I simply don't know how else to benchmark disk speeds. If you find an alternative do let me know please

[fulalas]

@rych, I have a Crucial MX500 SSD with 2 partitions: btrfs and ntfs. Here's the result:

Code: Select all

# hdparm -t /dev/sdb3

/dev/sdb3:
 Timing buffered disk reads: 1270 MB in  3.00 seconds = 423.26 MB/sec
# hdparm -t /dev/sdb1

/dev/sdb1:
 Timing buffered disk reads: 1230 MB in  3.00 seconds = 409.93 MB/sec
 

The first one is btrfs and the second is ntfs. The different is not huge, but still impressive.

[rych]

The different is not huge

fulalas, I think those are essentially the same value plus/minus. I'm suspicious about hdparm -- how does it measure what? It doesn't even check whether those partitions are mounted! I think it just measures the device overall speed class somehow.

I'm using dd and am beginning to see a pattern. But first, results on yet another machine. Again, 2 partitions ext4 and NTFS (mounted with ntfs-3g) on the same USB disk through 10Gbps port:

Writing:

Code: Select all

dd if=/dev/zero of=/mnt/sda3/tempfile bs=1M count=1024 conv=fdatasync |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.56873 s, 684 MB/s
dd if=/dev/zero of=/mnt/sda4/tempfile bs=1M count=1024 conv=fdatasync |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 3.36937 s, 319 MB/s

Reading:

Code: Select all

sudo sh -c "/usr/bin/echo 3 > /proc/sys/vm/drop_caches"
dd if=/mnt/sda3/tempfile of=/dev/null bs=1M count=1024 |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.58736 s, 676 MB/s
dd if=/mnt/sda4/tempfile of=/dev/null bs=1M count=1024 |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 4.73717 s, 227 MB/s

Conclusion: Off the same USB disk, under Linux, NTFS with ntfs-3g is twice slower than ext4, both reading and writing!?

[rych]

Tried my old (top of its class a couple of years ago, not cheap) Sandisk Extreme Pro USB, plugged into 5Gbps port, testing its two partitions: ext4 vs. NTFS mounted with ntfs-3g, and mounted with ntfs3. Writing/reading speed in MB/s are as follows:

8.4 vs. 6.6, and 7.2

350 vs. 272, and 348

So, I'm going to gradually introduce ntfs3 into NTFS mounting to observe its stability and integrity, before setting it as the automatic default for all NTFS partitions. But if it works, it's worth it: the speeds almost match those of ext4!

[fulalas]

Some people complaining it's still not reliable

https://www.reddit.com/r/archlinux/comm ... system_on/
https://forum.manjaro.org/t/ntfs3-keeps ... /33?page=1

[rych]

people complaining it's still not reliable

a few people were complaining -- yes, but months ago, well before the new fixes in 6.4 and 6.6 https://lore.kernel.org/ntfs3/. It's uncomfortable to be investigating this at frontier of hopefully first bug-free versions I agree My strategy is to enable it on the disk that's daily plugged into Windows and synched there -- Windows and my synchronization software will hopefully catch any file corruption. I'll post it then here and in NTFS3 Kernel Module

[rych]

... testing its two partitions: ext4 vs. NTFS mounted with ntfs-3g, and mounted with ntfs3. Writing/reading speed in MB/s are as follows:

trying now my fast NVMe Samsung SSD through the USB enclosure and the 10Gbps port:

538 vs. 428, and 551!

745 vs. 237, and 719!

Where before measuring ntfs3 speed, I remounted the NTFS partition with:

Code: Select all

umount /mnt/sda4
mount -t ntfs3 /dev/sda4

fulalas, how do you like those numbers?

[fulalas]

@rych, those numbers are impressive, indeed. But I'm wondering if you're using the most reliable test.

Have you considered using gnome-disks? It has an integrated benchmark. Give it a try and let me know what you think:

https://www.mediafire.com/file/vjclaty8 ... -1_SBo.xzm

[rych]

gnome-disks

Thanks for that nice utility. I've measured the reading speed of ntfs3, it gives an even higher number 851.8 MB/s (100 samples). That's more than I've ever seen from a USB3.1 10Gbps port.

Unfortunately, in order to perform its write test, the gnome-disks requires an unmounted access to the whole disk, not just that partition! But I can't unmount my /mnt/sda3 Porteus (or the /mnt/sda2 PorteuX partition) because I'm running on it!

[rych]

But I'm wondering if you're using the most reliable test

I also have the KDiskMark-3.1.3-fio-3.34-x86_64.AppImage. I personally don't trust it for its bugs? For example, one has to restart the program afresh when choosing a different disk/partition between tests -- otherwise it's just testing the previous partition again. Anyway here're the results that look about right:

KDiskMark (3.1.3): https://github.com/JonMagon/KDiskMark
Flexible I/O Tester (fio-3.34): https://github.com/axboe/fio
--------------------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

[Read]
Sequential 1 MiB (Q= 8, T= 1): 865.022 MB/s [ 844.7 IOPS] < 9437.22 us>
Sequential 128 KiB (Q= 32, T= 1): 785.451 MB/s [ 6136.3 IOPS] < 5202.63 us>
Random 4 KiB (Q= 32, T=16): 86.009 MB/s [ 21503.2 IOPS] < 5940.20 us>
Random 4 KiB (Q= 1, T= 1): 23.932 MB/s [ 5983.1 IOPS] < 166.23 us>

[Write]
Sequential 1 MiB (Q= 8, T= 1): 699.244 MB/s [ 682.9 IOPS] < 11394.37 us>
Sequential 128 KiB (Q= 32, T= 1): 656.682 MB/s [ 5130.3 IOPS] < 6211.55 us>
Random 4 KiB (Q= 32, T=16): 85.249 MB/s [ 21313.4 IOPS] < 5990.25 us>
Random 4 KiB (Q= 1, T= 1): 46.921 MB/s [ 11730.4 IOPS] < 83.97 us>

Profile: Default
Test: 1 GiB (x5) [Measure: 5 sec / Interval: 5 sec]
Date: 2023-11-01 21:59:25
OS: slackware 5.01 [linux 6.5.5-porteus]

Also during the intense testing the temperature went dangerously high (74 Celsius!) on the NVMe SSD inside, so I might indeed need to apply a thermal pad before putting it into the enclosure:

Code: Select all

root@porteus:~# smartctl -x -d sntrealtek /dev/disk/by-label/PorteuX | grep Temp*
Warning  Comp. Temp. Threshold:     85 Celsius
Critical Comp. Temp. Threshold:     85 Celsius
Temperature:                        55 Celsius
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               55 Celsius
Temperature Sensor 2:               74 Celsius

[rych]

By the way, the system static SSD still on ntfs-3g has these results (will be interesting to compare when I remount it with ntfs3)

KDiskMark (3.1.3): https://github.com/JonMagon/KDiskMark
Flexible I/O Tester (fio-3.34): https://github.com/axboe/fio
--------------------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

[Read]
Sequential 1 MiB (Q= 8, T= 1): 1037.815 MB/s [ 1013.5 IOPS] < 7808.20 us>
Sequential 128 KiB (Q= 32, T= 1): 839.224 MB/s [ 6556.4 IOPS] < 5015.91 us>
Random 4 KiB (Q= 32, T=16): 19.823 MB/s [ 4956.5 IOPS] < 25854.82 us>
Random 4 KiB (Q= 1, T= 1): 28.057 MB/s [ 7014.4 IOPS] < 141.95 us>

[Write]
Sequential 1 MiB (Q= 8, T= 1): 400.840 MB/s [ 391.4 IOPS] < 8382.00 us>
Sequential 128 KiB (Q= 32, T= 1): 407.736 MB/s [ 3185.4 IOPS] < 3536.56 us>
Random 4 KiB (Q= 32, T=16): 52.634 MB/s [ 13159.5 IOPS] < 9316.66 us>
Random 4 KiB (Q= 1, T= 1): 91.410 MB/s [ 22852.7 IOPS] < 26.89 us>

Profile: Default
Test: 1 GiB (x5) [Measure: 5 sec / Interval: 5 sec]
Date: 2023-11-01 22:25:21
OS: slackware 5.01 [linux 6.5.5-porteus]

UPDATE: The comparison is now done in the NTFS3 dedicated thread, NTFS3 Kernel Module (Post by rych #97030) and demonstrates significant speed improvements

[rych]

As RTL9210B using different firmware gave me a nasty stability issue on at least one machine (suddenly failing I/O, started to "spin" with flashing and increasing temperature, immediately freezing my Porteus killing keyboard and mouse)... I've decided to upgrade to this one:

Asmedia ASM2364 (20Gbps)

(my enclosure controller is ASM236X for some reason) -- of a higher class (20Gbps), but still plugged into a 10Gbps USB 3.1 port:

Code: Select all

root@porteus:~# lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 10000M
    |__ Port 2: Dev 3, If 0, Class=, Driver=uas, 10000M

This time I applied a thermal pad. The standby temperature is now considerably higher:

Code: Select all

root@porteus:~# smartctl -x -d sntasmedia /dev/disk/by-label/PorteuX | grep Temperature
Temperature:                        56 Celsius
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               56 Celsius
Temperature Sensor 2:               60 Celsius

(Needed a fresher smartctl to recognize "sntasmedia"). However it doesn't rise too much after some intensive testing at least: stays under 65 degrees.

Speed tests on its ext4 /mnt/sda3 partition:

KDiskMark (3.1.3): https://github.com/JonMagon/KDiskMark
Flexible I/O Tester (fio-3.34): https://github.com/axboe/fio
--------------------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

[Read]
Sequential 1 MiB (Q= 8, T= 1): 1015.669 MB/s [ 991.9 IOPS] < 8034.74 us>
Sequential 128 KiB (Q= 32, T= 1): 909.275 MB/s [ 7103.7 IOPS] < 4494.44 us>
Random 4 KiB (Q= 32, T=16): 96.824 MB/s [ 24207.1 IOPS] < 5282.25 us>
Random 4 KiB (Q= 1, T= 1): 25.087 MB/s [ 6271.7 IOPS] < 158.69 us>

[Write]
Sequential 1 MiB (Q= 8, T= 1): 774.676 MB/s [ 756.5 IOPS] < 10327.58 us>
Sequential 128 KiB (Q= 32, T= 1): 733.480 MB/s [ 5730.3 IOPS] < 5551.47 us>
Random 4 KiB (Q= 32, T=16): 98.027 MB/s [ 24507.9 IOPS] < 5213.59 us>
Random 4 KiB (Q= 1, T= 1): 41.485 MB/s [ 10371.3 IOPS] < 94.15 us>

Profile: Default
Test: 1 GiB (x5) [Measure: 5 sec / Interval: 5 sec]
Date: 2024-01-15 20:00:07
OS: slackware 5.01 [linux 6.6.3-porteus]

Speed tests on its NTFS, ntfs3 driver on /mnt/sda4 partition:

KDiskMark (3.1.3): https://github.com/JonMagon/KDiskMark
Flexible I/O Tester (fio-3.34): https://github.com/axboe/fio
--------------------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

[Read]
Sequential 1 MiB (Q= 8, T= 1): 1017.442 MB/s [ 993.6 IOPS] < 8017.45 us>
Sequential 128 KiB (Q= 32, T= 1): 907.701 MB/s [ 7091.4 IOPS] < 4501.56 us>
Random 4 KiB (Q= 32, T=16): 96.970 MB/s [ 24243.4 IOPS] < 5276.51 us>
Random 4 KiB (Q= 1, T= 1): 25.596 MB/s [ 6399.1 IOPS] < 155.56 us>

[Write]
Sequential 1 MiB (Q= 8, T= 1): 774.772 MB/s [ 756.6 IOPS] < 10300.75 us>
Sequential 128 KiB (Q= 32, T= 1): 734.296 MB/s [ 5736.7 IOPS] < 5555.16 us>
Random 4 KiB (Q= 32, T=16): 97.634 MB/s [ 24409.5 IOPS] < 5231.61 us>
Random 4 KiB (Q= 1, T= 1): 47.243 MB/s [ 11810.9 IOPS] < 83.20 us>

Profile: Default
Test: 1 GiB (x5) [Measure: 5 sec / Interval: 5 sec]
Date: 2024-01-15 20:06:34
OS: slackware 5.01 [linux 6.6.3-porteus]