[Moved] Enabling Porteus to boot from hybrid ISO-UDF

New features which should be implemented in Porteus; suggestions are welcome. All questions or problems with testing releases (alpha, beta, or rc) should go in their relevant thread here, rather than the Bug Reports section.
jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[Moved] Enabling Porteus to boot from hybrid ISO-UDF

Post#1 by jjr » 18 Nov 2023, 17:51

update-- as Rava notes, the mount ought to work because of fstab. The cause of the issue in finit may be unrelated (see added post).
--------------------------

Since this includes the fix, I wanted to post it separately somewhere for community visibility. I hope that's okay.

I'm using Porteus 5.0 for size constraints, but the same code is also in 5.01 (and probably other versions, but I haven't checked).

How I found this issue:

macOS cannot read files >4GB in size from any ISO9660 optical format. Apple has only ever supported ISO levels 1 and 2, not level 3 or ISO9660:1999, which added support for large files. (You can force-mount level 3, but large files will not be accessible.)

The workaround is to add "-udf" to the mkisofs command to create an ISO-UDF hybrid, which is recognizable as either format, and macOS can read UDF just fine. But I was stumped that Porteus was unable to boot from an ISO-UDF hybrid (neither via grub/syslinux/bios/uefi).

Extract the initrd.xz/cpio, and view the 'finit' file. Notice in this section there are 2 invalid 'mount' invocations:

Code: Select all

# Device partitions:
EOF
for x in $dev; do
    fs=`grep -w /dev/$x /tmp/devices | egrep -o ' TYPE=[^ ]+' | cut -d'"' -f2`
    [ $fs = vfat ] && echo "/dev/$x /mnt/$x vfat $MOPT,umask=0,check=s,utf8,uid=1000 0 0" >>/etc/fstab || echo "/dev/$x /mnt/$x $fs $MOPT 0 0" >>/etc/fstab
    if [ ! -d /mnt/$x ]; then
        mkdir /mnt/$x
        if [ $fs = ntfs ]; then
            ntfs-3g /dev/$x /mnt/$x -o $MOPT,uid=1000,big_writes 2>/dev/null || { sed -i "/$x /d" /etc/fstab; rmdir /mnt/$x; }
        else
            mount -n /mnt/$x 2>/dev/null || { modprobe $fs 2>/dev/null && mount -n /mnt/$x 2>/dev/null || { sed -i "/$x /d" /etc/fstab; rmdir /mnt/$x; }; }
        fi
    fi
done
Specifically:

Code: Select all

mount -n /mnt/$x 2>/dev/null || ... && mount -n /mnt/$x 2>/dev/null
Mount requires a device to be specified as well as a mountpoint.

At first glance, locate()'s later calls to mount_device() (which does invoke mount correctly) might appear to cover for those failed mounts, but alas! those get called only if the user specifies a from= cheatcode that hardcodes a "/mnt/" path or a UUID. Without either of those, the "search" case depends on the earlier mounts to have worked (which they didn't).

Anyway, if you fix that mount line under "# Device partitions":

Code: Select all

mount -n /mnt/$x 2>/dev/null || ... && mount -n /mnt/$x 2>/dev/null
to read this instead:

Code: Select all

mount -n /dev/$x /mnt/$x 2>/dev/null || ... && mount -n /dev/$x /mnt/$x 2>/dev/null
then ISO-UDF hybrids mount correctly, Porteus finds its filies, and macOS can access your Blu-Ray archives.

I see that FAT and NTFS are special cases (and so they avoid this problem),
but it looks as if this issue might affect booting from other alternative filesystems too.

If anyone needs to backport this fix to an earlier version, here's the way to extract and repack the initrd.xz:

Code: Select all

mkdir -p work/imgroot && cd work/imgroot
xz -dc < /mnt/Porteus/boot/syslinux/initrd.xz | cpio -idmv
# fix the finit file, then repack
find . 2>/dev/null | cpio -o -H newc -R root:root | xz -9 --format=xz --check=crc32 > ../initrd_new.xz
Last edited by jjr on 29 Nov 2023, 08:07, edited 2 times in total.

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[PATCH] Bug in finit prevents booting Porteus from alternative filesystems

Post#2 by jjr » 18 Nov 2023, 19:27

Forgot to mention, the reason why a hybrid hit this issue might have to do with blkid preferring the UDF interpretation:
https://bugs.launchpad.net/ubuntu/+sour ... ug/1022519
(but i guess it's irrelevant)

User avatar
Rava
Contributor
Contributor
Posts: 5416
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

[PATCH] Bug in finit prevents booting Porteus from alternative filesystems

Post#3 by Rava » 29 Nov 2023, 02:46

jjr wrote:
18 Nov 2023, 17:51
Mount requires a device to be specified as well as a mountpoint.
That is not always the case.

Example: Running Porteus 5.01 system.
I plug in an USB thumb drive. Drives available and mounted already are /dev/sda and /dev/sdb so the first free device will be /dev/sdc:

Code: Select all

root@rava:~# # Inserting the USB thumbdrive
root@rava:~# grep sdc /var/log/messages 
Nov 28 13:35:23 porteus kernel: sd 6:0:0:0: [sdc] 15656960 512-byte logical blocks: (8.02 GB/7.47 GiB)
Nov 28 13:35:23 porteus kernel: sd 6:0:0:0: [sdc] Write Protect is off
Nov 28 13:35:23 porteus kernel:  sdc: sdc1
Nov 28 13:35:23 porteus kernel: sd 6:0:0:0: [sdc] Attached SCSI removable disk
root@rava:~# mount /mnt/sdc1/  # mounting it using only the mount path
root@rava:~# df -Tm /mnt/sdc1/  # check it via df
Filesystem     Type 1M-blocks  Used Available Use% Mounted on
/dev/sdc1      vfat      7629  6200      1429  82% /mnt/sdc1
root@rava:~# umount /mnt/sdc1/  # umount it
root@rava:~# df -Tm /mnt/sdc1/  # checking the umounted mount path now shows the properties of / (the aufs of the live system) instead
Filesystem     Type 1M-blocks  Used Available Use% Mounted on
aufs           aufs       388   286       102  74% /
root@rava:~# mount /dev/sdc1  # mounting it only using the /dev/
root@rava:~# df -Tm /mnt/sdc1/  # check it via df
Filesystem     Type 1M-blocks  Used Available Use% Mounted on
/dev/sdc1      vfat      7629  6200      1429  82% /mnt/sdc1
root@rava:~# umount /mnt/sdc1/
root@rava:~# 
Using only the /dev/ or only the mount path is only possible because the other info it to be found in /etc/fstab :

Code: Select all

root@rava:~# grep sdc1 /etc/fstab 
/dev/sdc1 /mnt/sdc1 vfat users,noatime,nodiratime,suid,dev,exec,async,comment=x-gvfs-show,umask=0,check=s,utf8 0 0
root@rava:~# 
but the way you phrased it in your above post makes it look like you always need the /dev/ and the mount path.
When the device is not to be found in /etc/fstab (e.g. during early stages of the bootup) then you indeed need both the /dev/ and the target mount path to successfully mount a partition.
Cheers!
Yours Rava

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[updated] Bug in finit prevents booting Porteus from alternative filesystems

Post#4 by jjr » 29 Nov 2023, 08:06

Rava wrote:
29 Nov 2023, 02:46
Using only the /dev/ or only the mount path is only possible because the other info it to be found in /etc/fstab :
[...]
When the device is not to be found in /etc/fstab (e.g. during early stages of the bootup) then you indeed need both the /dev/ and the target mount path to successfully mount a partition.
Ah, I did not know that.

I'm confused then as to why adding the device to the mount command allowed it to work. :/

The trailing "||" clause of the vfat mount in the same block actually updates the fstab with an appropriate entry, which should match:

Code: Select all

    [ $fs = vfat ] && echo "/dev/$x /mnt/$x vfat $MOPT,umask=0,check=s,utf8,uid=1000 0 0" >>/etc/fstab || echo "/dev/$x /mnt/$x $fs $MOPT 0 0" >>/etc/fstab
I notice it's mandating $fs as the filesystem, and blkid reports an ISO-UDF hybrid as "udf", which is not available at boot.

There is a related issue that should be mentioned somewhere--

The modprobe command here, which would purport to load the UDF module, is actually a no-op due to a longstanding known BusyBox bug:

Code: Select all

            mount -n /mnt/$x 2>/dev/null || { modprobe $fs 2>/dev/null && mount -n /mnt/$x 2>/dev/null || { sed -i "/$x /d" /etc/fstab; rmdir /mnt/$x; }; }
You can check this in debug mode, as Porteus 5.0 uses BusyBox with the default CONFIG_DEFAULT_MODULES_DIR issue:
https://bugs.debian.org/cgi-bin/bugrepo ... bug=650159
...which results in xchdir aborting modprobe at this point (and printing the error message that you see in debug mode "can't change directory to ' ' "):
https://elixir.bootlin.com/busybox/1.35 ... obe.c#L572

(It concerns me it would appear that other modprobe commands in finit and linuxrc (5.0) are thus no-ops.)

This was fixed by Porteus 5.01 using the new BusyBox 1.36, which uses a sane default for CONFIG_DEFAULT_MODULES_DIR.
(So if you update the BusyBox binary in Porteus 5.0 and drop the UDF ko's into initrd, you can mount UDF...)

But even without the module, adding the device to the mount command allowed Porteus to boot from an ISO-UDF hybrid, and [thanks to your post] now I'm uncomfortable not knowing why... :(

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[updated] Bug in finit prevents booting Porteus from alternative filesystems

Post#5 by jjr » 29 Dec 2023, 08:08

Correction to the OP, can't edit:

Add -o $MOPT to the modified mount commands.

To summarize why (I surmise) the original fix works:

Explicitly specifying the device probably causes `mount` to bypass the fstab entry (whose $fs was forcing UDF), thus allowing it to autodetect the supported ISO9660 side of the hybrid.

The separate modprobe issue was a known bug in BusyBox 1.35 (Porteus 5.0),
but the lack of hybrid FS support wasn't really a bug in Porteus, just "lack of support".

So apologies for the thread title..

User avatar
Rava
Contributor
Contributor
Posts: 5416
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

[updated] Bug in finit prevents booting Porteus from alternative filesystems

Post#6 by Rava » 30 Dec 2023, 03:25

jjr wrote:
29 Dec 2023, 08:08
Correction to the OP, can't edit
[…]
So apologies for the thread title..
I could update / change the thread title if you want. Just tell me the title you want and I do it.
Cheers!
Yours Rava

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[updated] Bug in finit prevents booting Porteus from alternative filesystems

Post#7 by jjr » 31 Dec 2023, 02:08

Rava wrote:
30 Dec 2023, 03:25
jjr wrote:
29 Dec 2023, 08:08
Correction to the OP, can't edit
[…]
So apologies for the thread title..
I could update / change the thread title if you want. Just tell me the title you want and I do it.
OK. Ideally, it'd be less misleading to people / more truthful if moved to "Development" subforum with title something like:

[Moved] Enabling Porteus to boot from hybrid ISO-UDF

...Or at your discretion. Or if you can't move it, then just "[Not a bug] Enabling ......"

Thank you..

User avatar
Rava
Contributor
Contributor
Posts: 5416
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

[Moved] Enabling Porteus to boot from hybrid ISO-UDF

Post#8 by Rava » 31 Dec 2023, 20:05

jjr wrote:
31 Dec 2023, 02:08
moved to "Development" subforum with title something like:

[Moved] Enabling Porteus to boot from hybrid ISO-UDF
Done. :)

(Due to the forum glitch I had to log in several times to get it done. See here forum.porteus.org - forced logout, token outdated and unable to log-out )
Cheers!
Yours Rava

Post Reply