[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Please reproduce your error on a second machine before posting, and check the error by running without saved changes or extra modules (See FAQ No. 13, "How to report a bug"). For unstable Porteus versions (alpha, beta, rc) please use the relevant thread in our "Development" section.
User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 3941
Joined: 20 Mar 2012, 03:42
Distribution: v5.0-64bit
Location: australia
Contact:

[update:LILO/exfat] The generic Linux installer is incompatible with Ubuntu

Post#31 by ncmprhnsbl » 24 Feb 2024, 01:27

jjr wrote:
24 Feb 2024, 00:15
I don't know how Shellcheck misses this (it does!),
indeed, it's more concerned about the unquoted $(whoami) inside the test..but not about the garbage command ...are split usernames even allowed? quoted it anyway now.
another fixed again version:
:dl-green: Porteus-installer-for-Linux.com :dl-green: md5sum: 65dd04e797492b04aba34c40abba6753 (all the previous links are the same file)

one other thing i noticed looking at the makeself github issues was a possible case of /tmp being mounted in such a way that doesn't allow execution.
the simplist solution being a workaround by the user either using the --target /to/some/place or setting TMPDIR=/to/some/place , which could be mentioned in the docs.
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#32 by jjr » 24 Feb 2024, 02:34

Marking as solved again
ncmprhnsbl wrote:
24 Feb 2024, 01:27
one other thing i noticed looking at the makeself github issues was a possible case of /tmp being mounted in such a way that doesn't allow execution.
the simplist solution being a workaround by the user either using the --target /to/some/place or setting TMPDIR=/to/some/place , which could be mentioned in the docs.
Oh I didn't think of that. ;_;

Extracting to UnknownFS will always be a minefield. :(

that's what AppImage was designed to completely avoid (by mounting embedded image)...and I think Type 1/iso9660 is basically universal. But maybe not worth the effort... nntr

edit: Thank you again for all your work here :)

User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 3941
Joined: 20 Mar 2012, 03:42
Distribution: v5.0-64bit
Location: australia
Contact:

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#33 by ncmprhnsbl » 25 Feb 2024, 02:05

jjr wrote:
24 Feb 2024, 02:34
Thank you again for all your work here :)
you welcome :), but really, thank you, your solutions were the main driver in fixing this longstanding issue :worthy:
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#34 by jjr » 03 Mar 2024, 20:36

ncmprhnsbl wrote:
24 Feb 2024, 01:27
/tmp being mounted in such a way that doesn't allow execution.
jjr wrote:
24 Feb 2024, 02:34
minefield ... AppImage was designed to completely avoid (by mounting embedded image)
That probably made me sound like an idiot, since the AppImage itself has to be executable on FAT.

But it can be done 'cleanly' in sh the same way Makeself works.
Tested and working proof-of-concept below, with error checking and build commands.

Much tidier than the monstrous Makeself script.
But I'm sure you don't want to maintain a separate project like this..
NNTR

template.com (30 lines):

Code: Select all

#!/bin/sh
AUTORUN="./.porteus_installer/installer.com"
AUTORUN_ARGS=
MOUNTPOINT="/var/run/porteus_installer_mnt-$(cat /proc/sys/kernel/random/uuid)"  # /var/run = process lifetime scope; clears on reboot
SKIP=30
INSTALL_DIR="$(cd -- "$(dirname -- "$0")" > /dev/null 2>&1 ; pwd -P )"
SELF_PATH="$INSTALL_DIR/$(basename -- "$0")"
export INSTALL_DIR  # safer target than USER_PWD, to prevent user from accidentally modifying system disk
USER_PWD="$(pwd)"
export USER_PWD

fail() { echo >&2 "$1"; exit 1; }

# Validate environment
[ $(whoami) = 'root' ] || fail "This tool requires root privileges."
[ -f "$SELF_PATH" ] || fail "Failed to locate the script being run."
mkdir -p "$MOUNTPOINT" || fail "Failed to create mountpoint directory."

# Set trap to auto-unmount on script exit
trap "umount '$MOUNTPOINT' 2>/dev/null; rmdir '$MOUNTPOINT' 2>/dev/null" EXIT INT HUP TERM ABRT QUIT PIPE

# Mount the embedded image
OFFSET="$(head -n$SKIP "$SELF_PATH" | wc -c | tr -dC '0-9')"
mount -t iso9660 -o loop,exec,ro,offset=$OFFSET "$SELF_PATH" "$MOUNTPOINT" || fail "Mount command failed."
# Enter subshell to avoid holding the mountpoint open with chdir
(
	cd "$MOUNTPOINT" || fail "Couldn't chdir to image mountpoint."
	"$AUTORUN" $AUTORUN_ARGS "$@" || fail "Installer exited with error $?"
)
exit $?
#________ISO9660_ROCKRIDGE_IMAGE_DATA_CONTAINING_porteus_installer________
Build:

Code: Select all

ISOROOT="/path/to/dir/containing_.porteus_installer"
chmod -R +x "$ISOROOT"
mkisofs -o image.iso -rational-rock -no-pad -A IDATA -V IDATA "$ISOROOT"

LNUM=$(grep -n '^#________ISO9660' template.com | cut -d: -f1)
sed "s/^SKIP=.*/SKIP=$((LNUM-1))/g;/^#________ISO9660/d" template.com > Porteus-installer-for-Linux.com
cat image.iso >> Porteus-installer-for-Linux.com
Run:

Code: Select all

sudo sh Porteus-installer-for-Linux.com
edit: checksum validation is easy to add ofc
Last edited by jjr on 10 Mar 2024, 22:36, edited 13 times in total.

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#35 by jjr » 03 Mar 2024, 23:26

jjr wrote:
03 Mar 2024, 20:36
INSTALL_DIR=...
SELF_PATH=...
fixed :oops:
edit 2024/03/10: fixed (again) to support pathnames that start with "-"

also better exit code propagation and cleanup
Last edited by jjr on 10 Mar 2024, 22:36, edited 2 times in total.

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#36 by jjr » 04 Mar 2024, 22:42

Found out adding -no-pad reaches ~580KB (official is ~380KB), which stings less for the zero-extraction/safety benefit.

I'm sorry I am probably talking to myself.

User avatar
Ed_P
Contributor
Contributor
Posts: 8374
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#37 by Ed_P » 04 Mar 2024, 22:50

I enjoy seeing your attention to details jjr. :)
Ed

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#38 by jjr » 04 Mar 2024, 23:37

Ed_P wrote:
04 Mar 2024, 22:50
I enjoy seeing your attention to details jjr. :)
ty that was sweet of you :Rose:

User avatar
M. Eerie
Moderator
Moderator
Posts: 622
Joined: 31 Aug 2017, 21:18
Distribution: Nemesis Xfce/MATE x64

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#39 by M. Eerie » 05 Mar 2024, 11:36

jjr wrote:
04 Mar 2024, 22:42
I am probably talking to myself
I am following this thread with utmost attention. There is so much to learn from it...
> Does not compute_ 🖖

https://forum.porteus.org/viewtopic.php?p=94310#p94310
https://forum.porteus.org/viewtopic.php?p=84002#p84002
https://forum.porteus.org/viewtopic.php?p=77174#p77174
https://forum.porteus.org/viewtopic.php?f=39&t=8584

nanZor
Shogun
Shogun
Posts: 381
Joined: 09 Apr 2019, 03:27
Distribution: Porteus 5.01 x86-64 LXQT

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#40 by nanZor » 26 Mar 2024, 08:35

i think xfs is now supposed to be supported syslinux anyway.. does anyone actually used xfs any more? (apart from in a porteus save.dat)
I am right now with XFS as my changes partition. Along with a 500mb swapfile on it. Just simply cut my stick in half with fat32/xfs. Kind of reliving some old Knoppix days with reiserfs for it's "self healing" properties if someone yanks the stick out, or power outage kind of thing. Not so much worried about writes, especially with the EXIT: cheatcode. So far so good.

Thing is, I make my numerous Porteus sticks from within Porteus itself - total gparted party etc. But I've got Lubuntu on the internal drives of a few machines, so I'll be testing this out from the thread info!
That's a UNIX book - cool. -Garth

nanZor
Shogun
Shogun
Posts: 381
Joined: 09 Apr 2019, 03:27
Distribution: Porteus 5.01 x86-64 LXQT

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#41 by nanZor » 26 Mar 2024, 10:20

So I checked this out on Lubuntu 22.04 and see the craziness too. BUT!

Gparted Live also has an optional "copy n paste" zip file option, instead of an iso-burn - similar style of install that Porteus uses, and it too fails.

ncmprhnsbl - your modified installer above works on the Lubuntu test! I was able to make a bootable Porteus stick with your mod in Lubuntu! Nice.

Thing is - as super easy as a Porteus install is, the Lubuntu user would have to know how to use an archiver instead of mounting the iso. There is also a superflous [BOOT] directory showing up in the archiver along with regular boot directory. Then there is the stick being identified with UUID instead of the usual sdXx. AND, the user would have to know how to become root with the sudo -s method. No plain su or sudo root here.

On *buntu, they sure don't make installing Porteus to a stick easy, when it is the easiest thing to do!
That's a UNIX book - cool. -Garth

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#42 by jjr » 02 Apr 2024, 02:29

FYI, there's something weird/wrong going on in this installer.... :hmmm:

I was avoiding looking at the logic too deeply, partly because I didn't want to find more invasive work than the trivial fixes appropriate for an outsider. (Dumb logic on my part, maybe.)

But I'm adding SYSLINUX's GPT-BIOS support to the Windows and Linux installers for myself (for >2 TiB drives), so I "took mental ownership" of the installer / rewriting it with full review.

To summarize, the installer (both old and current 2024-02-23):
  • Relies on PRT being sometimes empty in order to avoid damaging user data.
  • Relies on PRT being always non-empty and valid.
  • Never, in fact, makes it possible for PRT to ever be empty (neither with the original definition, nor the current one).
Bottom-line: It would be rare for this to affect anyone. But I spent half the day trying to make heads or tails of it, so going to post anyway. In detail:
  1. It fails to avoid destroying device data when a user installs Porteus to a non-partitioned device, such as a whole-device XFS.

    Which is probably nobody. But the original code is intended to either (a) protect against corrupting a non-partitioned FS or (b) avoid overwriting a custom MBR:

    Notice it avoids burning the MBR to sector 0 if $PRT is empty:

    Code: Select all

    if [ "$PRT" ]; then
    	# Setup MBR:
    	dd if="$bin"/mbr.bin of="$DEV" bs=440 count=1 conv=notrunc >/dev/null 2>&1
    
    I don't know which one of the above was the author's intention (a or b), but it doesn't achieve either. Neither the original nor current definition of PRT could ever have been an empty string, so the test is useless.
    .
  2. This error check seems to be nonsense:

    Code: Select all

    DEV=/dev/$(lsblk -ndo pkname "$PRT")
    #...
    if [ -z "$(echo "$DEV" | xargs)" ]; then
    	echo "Current unit $MPT was not mounted correctly. ...
    
    Odd use of xargs aside(?), DEV will never be empty..
    .
  3. Another wrong error check, 'jeopardizing user data', which relies on PRT being empty:

    Code: Select all

    if [ -z "$PRT" ] && [ "$FS" = xfs ]; then
    	echo
    	echo "LILO cannot be installed on a device formatted with XFS as this"
    	echo "filesystem would be destroyed. Please create partition on $DEV"
    	...
    
----------------
I was going to say, if you wanted just to avoid corrupting a non-partitioned filesystem, you could use lsblk's behavior of returning an empty parent kernel device when PRT is not a partition.

You'd simply replace the faulty error check in location (2) with something like this:

Code: Select all

DEV="/dev/$(lsblk -ndo pkname "$PRT")"
if [ ! -b "$DEV" ]; then
	DEV="$PRT"
	PRT=""
fi
Then both of those tests for an empty PRT would make some sense. But there are problems with that:
  • Again, other code relies on PRT never being empty.
  • It would also allow a whole-disk filesystem to fall through into LILO/extlinux and (I presume) report that it "successfully" installed Porteus, when in fact the computer won't boot without an MBR. Bogus again.
(These could be anyone's fault -- or series of people -- who might've modified the code w/o noticing regressions. Things happen.)

[edit: snipped]

Anyway, the installer above (2024-02-23) should work as expected for practically everyone :Search: , so unless you want to ask/arbitrate fanthom's original intention and redesign to achieve it, you might as well leave it be...

Thank you for reading my large waste of time for today. :mellow: :worthy:
nntr...
Last edited by jjr on 02 Apr 2024, 07:37, edited 1 time in total.

User avatar
Ed_P
Contributor
Contributor
Posts: 8374
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#43 by Ed_P » 02 Apr 2024, 04:42

jjr wrote:
02 Apr 2024, 02:29
Thank you for reading my large waste of time for today.
:ROFL:
Ed

User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 3941
Joined: 20 Mar 2012, 03:42
Distribution: v5.0-64bit
Location: australia
Contact:

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#44 by ncmprhnsbl » 02 Apr 2024, 07:03

jjr wrote:
02 Apr 2024, 02:29
FYI, there's something weird/wrong going on in this installer.... :hmmm:

I was avoiding looking at the logic too deeply, partly because I didn't want to find more invasive work than the trivial fixes appropriate for an outsider. (Dumb logic on my part, maybe.)

But I'm adding SYSLINUX's GPT-BIOS support to the Windows and Linux installers for myself (for >2 TiB drives), so I "took mental ownership" of the installer / rewriting it with full review.
thanks for your continued interest in this :) GPT-BIOS support sounds like something others might find useful and "with full review" is welcome..
for a more complete history i'll post here the original (fanthoms work i presume) and the one you call 'old' ..
most of these changes were committed in one go (with the commit message: "Updated installer, included support to nvme devices") apart from the arch specific extlinux and root escalation parts and then the uuid injection bit.
this seems to be where at least some of the weirdness appears.
original installer.com:
newer pre your fixes installer.com
at the time my only review was to test that it 'worked'
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

jjr
Black ninja
Black ninja
Posts: 46
Joined: 18 Nov 2023, 17:10
Distribution: 5.0

[Solved] The generic Linux installer is incompatible with Ubuntu + fixing LILO/exFAT

Post#45 by jjr » 02 Apr 2024, 09:30

ncmprhnsbl wrote:
02 Apr 2024, 07:03
original installer.com:
Oh, wow!

So fanthom's original work actually had none of these issues -- nor even issue #1 from the Original Thread Post.

He's actually using df similar to how I suggested (in fact, his tail was a good idea, but my -P is what guarantees POSIX output format):

Code: Select all

PRT=`df -h . | tail -n1 | cut -d" " -f1`
Someone changed that to the broken cut-slash method that caused the bug:

Code: Select all

PRT=/dev/`echo $PWD | cut -d/ -f3`
Also the "$PRT" tests were originally "$PRTN" by him, which makes much more sense. (Someone changed them and broke it.) He also didn't have the meaningless $DEV test there.

However, someone (maybe same person) upgraded to using lsblk..pkname, which was a great improvement over his sed'ing to get DEV.

So I guess these versions should be merged, if only a little.
ncmprhnsbl wrote:
02 Apr 2024, 07:03
GPT-BIOS support sounds like something others might find useful and ...
OK, I'll aim to make a post eventually, when I'm comfortable with it.

For Linux it's straightforward. sfdisk, instead of setting the Active flag, must set the Legacy BIOS Bootable attribute that the syslinux gptmbr.bin (instead of mbr.bin) scans for, and that's basically it (besides testing the table type).

For Windows, I don't have a Win32 devel environment, so I had to hex-patch syslinux.exe swapping its embedded mbr.bin with gptmbr.bin and re-checksum. (Explained here.)

Then remove the -a (set Active) flag from inside Porteus-installer-for-Windows-GPT.exe.

But it seemed unwise to try to script diskpart in Batch to set the Legacy BIOS Bootable attribute, esp. since the WMIC command-line utility to get disk#/part# is deprecated as of Windows 10, Microsoft pushing everyone to PowerShell to access WMI.

So the user would have to set the attribute manually (in diskpart), which Microsoft pretends doesn't exist, but it does work if you add 15 zeros:

Code: Select all

gpt attributes=0x0000000000000005
So not very user-friendly. :hmmm:

[edit: snipped.. some incorrect info]

Post Reply