Booting from NVMe USB enclosure
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
Does Porteus 5.0 (with the stock or updated kernel, or PorteuX) recognize and boot from an NVMe SSD in a USB 3.1 enclosure, with one of the following interface chipsets (10Gbps):
* Realtek RTL9210B or RTL9210
* JMicron JMS583 A3
* Asmedia ASM2362 or even Asmedia ASM2364 (20Gbps)
Does anyone have any experience with those? Which chipset would be better?
The enclosure will be carrying a Samsung 980 Samsung 970 EVO Plus 1TB M.2 PCIe 3.0 NVMe Internal SSD. It's on sale but still not cheap. So need some advice first. I hope this works and I get as much speed out of my USB 3.1 as possible.
The motherboard/BIOS clearly support booting the Windows off the internal system disk which is also an NVMe PCIe disk. So I guess it's a matter of grub2 or a porteus native boot loader seeing and capable of booting off a similar disk through the USB interface?
* Realtek RTL9210B or RTL9210
* JMicron JMS583 A3
* Asmedia ASM2362 or even Asmedia ASM2364 (20Gbps)
Does anyone have any experience with those? Which chipset would be better?
The enclosure will be carrying a Samsung 980 Samsung 970 EVO Plus 1TB M.2 PCIe 3.0 NVMe Internal SSD. It's on sale but still not cheap. So need some advice first. I hope this works and I get as much speed out of my USB 3.1 as possible.
The motherboard/BIOS clearly support booting the Windows off the internal system disk which is also an NVMe PCIe disk. So I guess it's a matter of grub2 or a porteus native boot loader seeing and capable of booting off a similar disk through the USB interface?
Last edited by rych on 24 Apr 2023, 08:55, edited 3 times in total.
- Ed_P
- Contributor
- Posts: 8361
- Joined: 06 Feb 2013, 22:12
- Distribution: Cinnamon 5.01 ISO
- Location: Western NY, USA
Booting from NVMe USB enclosure
Sounds reasonable to me. What are you using to boot the USB drive, grub2 or some app?
Ed
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
Yes, I use grub2 instead of the native porteus boot loader. The question is which chip will work guaranteed?
(I'm leaning toward Realtek RTL9210B. The newer Asmedia ASM2364 will support up to 20Gbps though. Not that I have such USB ports yet. Besides, all that speed doesn't matter, if it won't boot reliably for example)
- Ed_P
- Contributor
- Posts: 8361
- Joined: 06 Feb 2013, 22:12
- Distribution: Cinnamon 5.01 ISO
- Location: Western NY, USA
Booting from NVMe USB enclosure
I've never tried a USB drive with the options you are considering so I can't suggest one over another. They sound impressive though and will be interested in your experience.
If you partition the drive to have a FAT32 partition you have the option for UEFI booting otherwise you're limited to BIOS booting. These are the two grub2 menus I use for booting my USB drives, which are all less then 32GB and formated as FAT32.
If you partition the drive to have a FAT32 partition you have the option for UEFI booting otherwise you're limited to BIOS booting. These are the two grub2 menus I use for booting my USB drives, which are all less then 32GB and formated as FAT32.
Code: Select all
menuentry " Porteus 5.0 USB - EFI" --class slackware --class icon-porteus {
set bootmgr=/EFI/boot/bootx64.efi #grubx64.efi # bootx64.efi
set bootdrv=$root
search -f $bootmgr --set=root
echo bootmgr: $bootmgr
sleep -v -i 3
if [ $root != $bootdrv ]; then
chainloader $bootmgr
else
echo "----------------------------------------"
echo USB drive NOT found.
echo
sleep -v -i 10
fi
set root=$bootdrv
}
menuentry " Porteus 5.0 USB - AF'" --class slackware --class icon-porteus {
set porteus_parms="volume=33 reboot=cold extramod=/Modules" #;cinnamon" # changes=EXIT:/changes/porteussave.dat"
set bootdrv=$root
search -f /boot/syslinux/vmlinuz --set=root
if [ $root != $bootdrv ]; then
linux /boot/syslinux/vmlinuz $porteus_parms
initrd /boot/syslinux/initrd.xz
else
echo "----------------------------------------"
echo USB drive NOT found.
echo
sleep -v -i 10
fi
set root=$bootdrv
}
Ed
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
OK, I've ordered the above SSD and will be choosing my enclosure next. So it'd better work. What I'm after is the 10Gbps class speed. You see below how my best-in-class (SanDisk Extreme Pro 120GB) USB disk only claims up to 5Gbps out of the available on my system USB 3.1 gen 2 hub throughput of 10Gbps?
Code: Select all
lsusb -t | egrep -i "storage" -B1
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 10000M
|__ Port 1: Dev 2, If 0, Class=, Driver=usb-storage, 5000M
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
My Realtek RTL9210B enclosure carrying a Samsung 970 EVO Plus 1TB M.2 PCIe 3.0 NVMe Internal SSD when plugged into the USB-C port (!) reports 10000M bandwidth:
That's a new line in that output with "Driver=uas", the preceding "usb-storage" is my ordinary USB 3.0 SanDisk Extreme Pro 120GB disk which currently runs Porteus and is only up to 5000M
Next I'm going to test some actual speeds of the USB 3, the new USB-C, and the system SSD.
Code: Select all
root@porteus:~# lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 10000M
|__ Port 1: Dev 2, If 0, Class=, Driver=usb-storage, 5000M
|__ Port 2: Dev 8, If 0, Class=, Driver=uas, 10000M
Next I'm going to test some actual speeds of the USB 3, the new USB-C, and the system SSD.
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
I know disk benchmarking is a science in itself but using hdparm -t to check Read speed of these 3 disks does show that the new disk (/dev/sda) outperforms tboth the old USB (fastest pendrive as it were) and the system SSD!!
See also: https://www.everythingusb.com/speed.html
1. Samsung 970 EVO Plus 1TB M.2 PCIe 3.0 NVMe (rated 3,500MB/s) in Realtek RTL9210B USB 3.1 enclosure (10Gbs)
2. SanDisk Extreme Pro (5Gbs, upto 420MB/s)
3. System's SSD, NVMe SK hynix (upto 2200 MB/s)
Code: Select all
root@porteus:~# hdparm -t /dev/sda
/dev/sda:
Timing buffered disk reads: 2502 MB in 3.00 seconds = 833.99 MB/sec
root@porteus:~# hdparm -t /dev/sdb
/dev/sdb:
Timing buffered disk reads: 1136 MB in 3.00 seconds = 378.36 MB/sec
root@porteus:~# hdparm -t /dev/nvme0n1p3
/dev/nvme0n1p3:
Timing buffered disk reads: 1798 MB in 3.00 seconds = 598.74 MB/sec
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
Code: Select all
root@porteus:~# hdparm -t --direct /dev/sda
/dev/sda:
Timing O_DIRECT disk reads: 2214 MB in 3.00 seconds = 737.83 MB/sec
root@porteus:~# hdparm -t --direct /dev/sdb
/dev/sdb:
Timing O_DIRECT disk reads: 1038 MB in 3.00 seconds = 345.96 MB/sec
root@porteus:~# hdparm -t --direct /dev/nvme0n1p3
/dev/nvme0n1p3:
Timing O_DIRECT disk reads: 1916 MB in 3.00 seconds = 638.55 MB/sec
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
High speed may cause critical temperatures. I didn't apply a thermal pad to the NVMe before inserting it into the enclosure relying on the thin air. So l'm going to be checking the temperature:
Code: Select all
root@porteus:~# smartctl -x -d sntrealtek /dev/disk/by-label/PorteuX | grep Temp*
Warning Comp. Temp. Threshold: 85 Celsius
Critical Comp. Temp. Threshold: 85 Celsius
Temperature: 30 Celsius
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Temperature Sensor 1: 30 Celsius
Temperature Sensor 2: 33 Celsius
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
This setup is now stable (and fast!). I had to update the controller firmware with one important setting to completely disable the sleep/standby/poweroff of the idle NVMe SSD disk. Usually a desirable feature in hope to save some (small) power and to keep temperature low (it's already low); but for our purposes of running Porteus off this enclosure we actually want to prevent it from going to sleep in order to avoid stability issues: on some machines, when it goes into sleep, it often can't wake up properly leading to the system freeze with the only terrible remedy being the button-restarting the computer. Now it's fine and I can recommend this setup as the fastest/largest/affordable USB Porteus system.
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
I spoke too soon. The mundane USB cable that came with the enclosure quickly wore out and started only allowing USB 2 speeds suddenly. I had to invest in a good thick and short 20Gbps USB-C 3.2 cable at about the same price as the enclosure itself, so no wonder. Now it's back to its max speed and reliability. One of the cable ends is L-shape plugging nicely along the enclosure side instead of wildly sticking out of it perpendicularly.
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
Other ways to test read-write speeds: https://www.baeldung.com/linux/disk-performance-test
Code: Select all
dd if=/dev/zero of=/tmp/tempfile bs=1M count=1024 conv=fdatasync
sudo sh -c "/usr/bin/echo 3 > /proc/sys/vm/drop_caches"
dd if=/tmp/tempfile of=/dev/null bs=1M count=1024
-
- Shogun
- Posts: 238
- Joined: 12 Jun 2021, 21:59
- Distribution: Xfce 4.12 - 5.rc3 - x86_64
- Location: France
Booting from NVMe USB enclosure
Very interesting !rych wrote: ↑23 Oct 2023, 08:50Other ways to test read-write speeds: https://www.baeldung.com/linux/disk-performance-testCode: Select all
dd if=/dev/zero of=/tmp/tempfile bs=1M count=1024 conv=fdatasync sudo sh -c "/usr/bin/echo 3 > /proc/sys/vm/drop_caches" dd if=/tmp/tempfile of=/dev/null bs=1M count=1024
Last edited by Rapha_ on 25 Oct 2023, 10:35, edited 1 time in total.
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
Here are my tests on this machine. Both the host system disk and the USB enclosed disk are NVMe SSD. The partitions are:
sda3: ext4 on USB
sda4: NTFS on USB
nvme0n1p3: NTFS host system
The NTFS driver is the FUSE ntfs-3g, hence perhaps slower than expected speeds. Unless that's the limitation of my system disk, see the bottom of this post. Anyways, will be interesting to compare when I learn how to remount my NTFS partitions with the kernel driver ntfs3.
Writing:
Code: Select all
dd if=/dev/zero of=/tmp/tempfile bs=1M count=1024 conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.299371 s, 3.6 GB/s
dd if=/dev/zero of=/mnt/sda3/tempfile bs=1M count=1024 conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.87841 s, 572 MB/s
dd if=/dev/zero of=/mnt/sda4/tempfile bs=1M count=1024 conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.21219 s, 485 MB/s
dd if=/dev/zero of=/mnt/nvme0n1p3/tempfile bs=1M count=1024 conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.51177 s, 427 MB/s
Code: Select all
sudo sh -c "/usr/bin/echo 3 > /proc/sys/vm/drop_caches"
dd if=/tmp/tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.126275 s, 8.5 GB/s
dd if=/mnt/sda3/tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.35096 s, 795 MB/s
dd if=/mnt/sda4/tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 4.7413 s, 226 MB/s
dd if=/mnt/nvme0n1p3/tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.11127 s, 966 MB/s
Code: Select all
hdparm -t --direct /dev/sda
/dev/sda:
Timing O_DIRECT disk reads: 2182 MB in 3.00 seconds = 726.90 MB/sec
hdparm -t /dev/sda
/dev/sda:
Timing buffered disk reads: 2300 MB in 3.05 seconds = 755.00 MB/sec
hdparm -t --direct /dev/nvme0n1
/dev/nvme0n1:
Timing O_DIRECT disk reads: 1998 MB in 3.00 seconds = 665.79 MB/sec
hdparm -t /dev/nvme0n1
/dev/nvme0n1:
Timing buffered disk reads: 1856 MB in 3.00 seconds = 618.38 MB/sec
Sequential Read 1,382 MBytes/Sec
Sequential Write 579 MBytes/Sec
-
- Warlord
- Posts: 622
- Joined: 04 Jan 2014, 04:27
- Distribution: Porteus 5.0 x64 OpenBox
- Location: NZ
- Contact:
Booting from NVMe USB enclosure
I'm on a slightly different machine today. The system disk is a SATA SSD with an NTFS partition on sda3. The USB enclosed NVMe SSD is on sdb3 for ext4 partition, and sdb4 -- NTFS partition. The USB port is 5Gbit/s this time, so twice slower throughput than the 10Gbit/s USB.
We'll pipeline the output to the last line only (or, `2> >(grep copied)`)
Writing:
Reading after the mandatory :
Looks like
1. writing on the system SSD NTFS is much slower than expected
2. reading from an NTFS partition on the USB is much slower than reading from the adjacent ext4 partition
(By the way, dd has `oflag=direct` and `iflag=direct` options to bypass caching?)
We'll pipeline the output to the last line only (or, `2> >(grep copied)`)
Writing:
Code: Select all
dd if=/dev/zero of=/tmp/tempfile bs=1M count=1024 conv=fdatasync |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.33387 s, 3.2 GB/s
dd if=/dev/zero of=/mnt/sda3/tempfile bs=1M count=1024 conv=fdatasync |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.8858 s, 67.6 MB/s
dd if=/dev/zero of=/mnt/sdb3/tempfile bs=1M count=1024 conv=fdatasync |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.49062 s, 431 MB/s
dd if=/dev/zero of=/mnt/sdb4/tempfile bs=1M count=1024 conv=fdatasync |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.60756 s, 412 MB/s
Code: Select all
sudo sh -c "/usr/bin/echo 3 > /proc/sys/vm/drop_caches"
Code: Select all
dd if=/tmp/tempfile of=/dev/null bs=1M count=1024 |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.169581 s, 6.3 GB/s
dd if=/mnt/sda3/tempfile of=/dev/null bs=1M count=1024 |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.09434 s, 513 MB/s
dd if=/mnt/sdb3/tempfile of=/dev/null bs=1M count=1024 |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.40742 s, 446 MB/s
dd if=/mnt/sdb4/tempfile of=/dev/null bs=1M count=1024 |& tail -1
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.42216 s, 127 MB/s
Code: Select all
hdparm -t /dev/sda3
/dev/sda3:
Timing buffered disk reads: 1352 MB in 3.00 seconds = 450.25 MB/sec
hdparm -t /dev/sdb3
/dev/sdb3:
Timing buffered disk reads: 1196 MB in 3.00 seconds = 398.64 MB/sec
1. writing on the system SSD NTFS is much slower than expected
2. reading from an NTFS partition on the USB is much slower than reading from the adjacent ext4 partition
(By the way, dd has `oflag=direct` and `iflag=direct` options to bypass caching?)