Porteus USB live install

[robang74]

Porteus Linux live USB bootable installer

It creates a bootable USB with 512MB of persistence

https://github.com/robang74/porteus-usb-installer

Warning: it still under development. Just for feedbacks.

[Ed_P]

Interesting. Does it follow the steps in the USB_INSTALLATION.txt file? Is it a copy of Porteus' Porteus Installer app? Does it run on Windows? Might help some new Linux users.

Added in 12 minutes 47 seconds:
And can it work for PorteuX also.

Added in 27 minutes 57 seconds:
Just viewed the .sh script. Impressive work.

[Kulle]

Is the script trustworthy?
Here the script:

Code: Select all

#!/bin/bash
#
# (C) 2025, Roberto A. Foglietta <[email protected]> - 3-clause BSD
#
################################################################################
set -e

# Comment this line below to have the journal within the persistence loop file
journal="-O ^has_journal"

# This below is the default size in 512-blocks for the persistent loop file
blocks="1M"

################################################################################

function perr() {
    { echo; echo "$@"; } >&2
}

function usage() {
    perr "USAGE: bash $shs /path/file.iso [/dev/]sdx [it]"
    exit 1
}

function missing() {
    perr "ERROR: file '${1:-}' is missing, abort!"
    exit 1
}

function sure() {
    local ans
    echo
    read -p "Are you sure to contine [N/y] " ans
    echo
    test "$ans" == "Y" -o "$ans" == "y" && return 0
    exit 1 
}

function waitdev() {
    partprobe
    for i in $(seq 1 100); do
        egrep " $1$" /proc/partitions && return 0
        sleep 0.1
    done
    perr "ERROR: waitdev('$1') failed, abort!"
    exit 1
}

function mke4fs() {
    local lbl=$1 dev=$2; shift 2
    mkfs.ext4 -L $lbl -E lazy_itable_init=1,lazy_journal_init=1 -F $dev "$@"
}

wdr=$(dirname $0)
shs=$(basename $0)

if [ "$(whoami)" != "root" ]; then
    perr "WARNING: the script '$shs' requires root priviledges"
    sudo bash $0 "$@"
    exit $?
fi

################################################################################

iso=${1:-}
dev=${2:-}
kmp=${3:-}

sve="changes.dat"
bsi="moonwalker-bootscreen.png"
opt="-E lazy_itable_init=1,lazy_journal_init=1 -F"
cfg="/boot/syslinux/porteus.cfg"
mbr="porteus-usb-bootable.mbr.gz"
dst="/tmp/d"
src="/tmp/s"

test -b "/dev/$dev" || dev=$(basename $dev)
test -b "/dev/$dev" || usage
test -r "$iso" || iso="$wdr/$iso"
test -r "$iso" || usage

test -r "$bsi" || bsi="$wdr/$bsi"
test -f "$bsi" || bsi=""
test -r "$mbr" || mbr="$wdr/$mbr"
test -r "$mbr" || missing "$mbr"
test -n "$kmp" && kmp="kmap=$kmp"

trap "echo;echo;exit 1" INT

perr "RUNNING: $shs $(basename $iso) into /dev/$dev" ${kmp:+with $kmp}
echo; fdisk -l /dev/${dev} | sed -e "s,^.*$,\t&,"
perr "WARNING: all data on '/dev/$dev' will be LOST"
sure

# Clear previous failed runs, eventually
umount ${src} ${dst} 2>/dev/null || true
umount /dev/${dev}? 2>/dev/null || true
declare -i tms=$(date +%s%N)

# Write MBR and basic partition table
zcat ${mbr} >/dev/${dev}; waitdev ${dev}1

# Prepare partitions and filesystems
mkfs.vfat -n Porteus /dev/${dev}1
printf "n\np\n2\n\n\nw\n" | fdisk /dev/${dev}; waitdev ${dev}2
mke4fs "Portdata" /dev/${dev}2

# Mount source and destination devices
mkdir -p ${dst} ${src}; mount /dev/${dev}1 ${dst}
mount -o loop ${iso} ${src}

# Copying Porteus files from ISO file
cp -arf ${src}/* ${dst}
sync -f ${dst}${cfg} &

# Creating persistence loop filesystem
dd if=/dev/zero count=1 seek=${blocks} of=${sve}
mke4fs "changes" ${sve} ${journal}

# Moving persistence and configure it
perr "INFO: waiting for fsdata synchronisation..."
wait
test -r ${dst}${cfg} || missing ${dst}${cfg}
if test -n "${bsi}" && cp -f ${bsi} ${dst}/boot/syslinux/porteus.png;
   then echo "INFO: custom boot screen background '${bsi}' copied"; fi
sed -e "s,APPEND changes=/porteus$,&/${sve} ${kmp}," -i ${dst}${cfg}
grep -n "changes=/porteus/${sve}" ${dst}${cfg}
mv -f ${sve} ${dst}/porteus/

# Umount source and eject USB device
perr "INFO: waiting for umount synchronisation..."
umount ${src} ${dst}
eject /dev/${dev}
set +xe

# Say goodbye and exit
echo
let tms=($(date +%s%N)-$tms+500000000)/1000000000
echo "INFO: Installation completed in $tms seconds"
echo
echo "DONE: bootable USB key ready to be removed safely"
echo

[robang74]

Is the script trustworthy?

It is open source, thus you can check it by your own eyes and test in a live distro, just in case.

I did not tried it on Porteus itself but I will soon. So, if you are unsure then wait a little.

Soon you will be able to test the script within the Porteus live: 100% safe to test.

Just viewed the .sh script. Impressive work.

Thanks.

[Ed_P]

I did not tried it on Porteus itself but I will soon.

Won't it be best to post about a script proven to work for you? Looking forward to what you are developing.

[robang74]

Won't it be best to post about a script proven to work for you?

What are you talking about? I am THE developer of that script and I am testing with my root account of my main Ubuntu system. A mistake and I might wipe all my stuff!

Do you think I did not test it **enough**?

Sure, I tested it but I did not tested that it works also WITHIN a porteus live.

Moreover, the script, requires `bash` because it uses bashims. This should change in future, because I want to make it working also with busybox a/sh.

Plus, before moving to try it on different systems than my laptop, I wish to create a net-installer that download all my stuff and the ISO and do the magic. [1]

POST SCRIPTUM

[1] This make sense because Porteus has the all-in-memory mode so we can download the ISO and write it directly on as USB

Code: Select all

wget -O- $url | sudo dd bs=1M of=/dev/sdb 

The put that USB into a laptop/PC, boot in all-in-memory mode, clone the git and re-write that USB within a safe environment.

Which is the reason because `dd` uses the `seek` option creating a virtual file that does not waste space, in evoluted filesystem that allows holes into files, at least.

The when that file is copied into the VFAT32, then the holes are fulfilled of zeros, hopefully.

In fact, `cp` is supposed to not access on underlying physical device unless the kernel allocated blocks into the file instead of holes.

This implies that we do not need 512Mb (or 128Mb per the last v0.2.5 version) of memory to create that files but much lesser (c.a. 10Mb).

While /boot /syslinix and /porteus folders are - supposedly - loaded in RAM, so we can rewrite the USB stick reading from the RAM filesystem.

[robang74]

Porteus 5.01 Mate booting from an old SanDisk Cruzer Fit 32GB USB 2.0 and running on an ultra-old Thinkpad X220 equipped with 4GB of RAM. The storage stick is inserted in the USB 2.0 port, on the bottom left angle, and it is so short that it cannot even be seen without checking for it specifically for it.



How long does it take to prepare such a USB 2.0 stick? Less than 120 seconds with 128MB of persistence. Almost all the time is spent in writing the USB, hence that matters is the writing speed and data size.

https://github.com/robang74/porteus-usb-installer

[Ed_P]

What are you talking about? I am THE developer of that script and I am testing with my root account of my main Ubuntu system. A mistake and I might wipe all my stuff!

You're the one who said:

I did not tried it on Porteus itself

But in that regard, a script that supports a non-porteus user installing a non-installed Porteus ISO and creating a bootable Porteus USB drive would indeed be useful. And impressive.

Added in 1 hour 12 minutes 10 seconds:
My test:

Code: Select all

guest@porteus:~$ ./USBinstaller.sh
total 4.0M
srwxrwxrwx 1 1000    0 Mar 20 16:52 dbus-XjfvhIh1KA
-rw-r--r-- 1 1000 3.7M Mar 20 16:57 eb968a1a-355d-4ff3-b5d6-35548db4e325.zip
-rwxr-xr-x 1 1000 291K Mar 20 17:21 usbinstall.sh

/tmp/usbinstall.sh: line 7: syntax error near unexpected token `newline'
/tmp/usbinstall.sh: line 7: `<!DOCTYPE html>'
guest@porteus:~$ 

My .sh script:

Code: Select all

#!/bin/sh

# https://forum.porteus.org/viewtopic.php?p=102389

# wget download-url -O output-location-and-filename  --no-use-server-timestamps

wget https://github.com/robang74/porteus-usb-installer/blob/main/porteus-usb-install.sh -O  /tmp/usbinstall.sh   2>/dev/null
chmod +x /tmp/*sh
ls -hon /tmp
read
sh /tmp/usbinstall.sh 

It always downloads the current version. I'm running Porteus 5.01 Cinnamon.

Added in 9 minutes 25 seconds:
Oh crap. The URL I used is for the WHOLE webpage not just the .sh code. $#@%^

[robang74]

But in that regard, a script that supports a non-porteus user installing a non-installed Porteus ISO and creating a bootable Porteus USB drive would indeed be useful.

For sure!



About downloading from github there two main (simple) ways:

---

1. Using git to clone the repo and git pull to update it.

Code: Select all

git clone https://github.com/robang74/porteus-usb-installer.git
cd porteus-usb-installer
git pull # only when you want to sync with the repo

---

2. use the tagged version and download a .zip file

Go with the browser to the tagged archive repository

https://github.com/robang74/porteus-usb-installer/tags

Download the last version, unzip, enter in the directory

---

2bis. you know the last tagged version link and you use it

Code: Select all

wget https://github.com/robang74/porteus-usb-installer/archive/refs/tags/v0.2.5.tar.gz
mkdir porteus-usb-install-v0.2.5
tar xvzf v0.2.5.tar.gz -C porteus-usb-install-v0.2.5
cd porteus-usb-install-v0.2.5

---

At this point, you need to have the ISO or download it.

Usage: bash porteus-usb-install.sh /path/file.iso [/dev/]sdx [it]

I suggest you to use the browser to download, the wget -O- example isn't supposed to work in such a way. But in this way

Code: Select all

url=https://linux.rz.rub.de/porteus/x86_64/current/Porteus-MATE-v5.01-x86_64.iso
wget -O- $url | sudo dd bs=1M of=/dev/sdb
sudo eject /dev/sdb

At this point, you can boot your machine in RAMFS mode (everything on RAM) and do all the magics but my script it is not ready for that, yet.

Is the wget -O- way the best one? Nope, it was just for bragging!

...and in case if you like to know which was the counter-bragging answer...

pxe boot directly from the url!

[robang74]

In the v0.2.6, I have added an extra safety check. Usually the system root (/) cannot be unmounted and trying to use a device that has a partition mounted generates a failure. This should prevent that people might destroy their system/data by mistake. However, this safety check might fail in case someone has an external data disk attached which is not busy and thus can be unmounted. However, to prevent this scenario, the script ask the user to agree with the device writing:

Code: Select all

RUNNING: porteus-usb-install.sh Porteus-MATE-v5.01-x86_64.iso into /dev/sdb with kmap=it

	Disk /dev/sdb: 114.6 GiB, 123048296448 bytes, 240328704 sectors
	Disk model:  SanDisk 3.2Gen1
	Units: sectors of 1 * 512 = 512 bytes
	Sector size (logical/physical): 512 bytes / 512 bytes
	I/O size (minimum/optimal): 512 bytes / 512 bytes
	Disklabel type: dos
	Disk identifier: 0x25c632fa
	
	Device     Boot Start    End Sectors  Size Id Type
	/dev/sdb1  *        0 876543  876544  428M 17 Hidden HPFS/NTFS

WARNING: all data on '/dev/sdb' will be LOST

Are you sure to continue [N/y] y 

The fdisk output si providing enough information to the user to understand which device is going to be written. However, if your data device is almost identical...

...well... sh*t might happen! But at least everything was reasonable to prevent it, it was done.

As per general rule: detach EVERY device that matters for you BEFORE using `dd` or any script or apps that uses `dd` or anything similar with root privileges.

Linux is a very powerful system but like every powerful tools, do not forgive errors.

[Ed_P]

This approach working so far at this point.

Code: Select all

#!/bin/sh

# https://forum.porteus.org/viewtopic.php?p=102389

# https://github.com/robang74/porteus-usb-installer/tags  ***** Note Current version

if [ `whoami` != "root" ]; then
   echo "Enter root's password" $(tput bold)$(tput setaf 2)  # 1-red, 2-green, 6-cyan, 7-white
   su -c "sh $0 $1 $2"; exit
fi
echo $(tput sgr0) 

wget https://github.com/robang74/porteus-usb-installer/archive/refs/tags/v0.3.6.zip -O /tmp/git.zip  2>/dev/null
7z x -y /tmp/git.zip -o/tmp 2>/dev/null
cd /tmp/porteus-usb-installer*
chmod +x *sh
ls -hon 
read
echo "porteus-usb-install.sh" && echo " "
sh porteus-usb-install.sh 

And starts this:

Code: Select all

-rwxr-xr-x 1 1000 4.1K Mar 20 18:05 porteus-usb-install.sh

WARNING: the script 'porteus-usb-install.sh' requires root priviledges

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

For security reasons, the password you type will not be visible.

Password: 
sudo: timed out reading password
sudo: a password is required
guest@porteus:~$ 

As per general rule: detach EVERY device that matters for you BEFORE using `dd` or any script or apps that uses `dd` or anything similar with root privileges.

YES And why my next posting will be much later.

[robang74]

This approach working so far at this point.

This `sh *.sh` is a VERY BAD way of proceeding. IMHO.

---

This script, called in develop mode download an ISO, the last MATE for x86_64 by default

Code: Select all

wget https://raw.githubusercontent.com/robang74/porteus-usb-installer/refs/heads/main/porteus-net-install.sh
sudo -k; DEVEL=1 bash porteus-net-install.sh [/dev/sdX] [it]

checks its sha256sum, download the main HEAD about scripts and create the USB disk on /dev/sdX.

Enjoy, R-

[ncmprhnsbl]

nice
some suggestions:
add a "sanity check", ie. check for the necessary utilities on the host system, say, with which (i know this stuff is pretty much everywhere, but it doesn't hurt to check)
i notice you include the keymap cheatcode, this can be used in the /porteus/porteus-v5.0-x86_64.cfg file, which would make it work across all boot entries.
same for timezone= cheat, which might be nice to include as an option.

question: am i correct that two partitions are created 1st. fat32, 2nd. ext4 ?
if that's the case, it's possible to place the {boot,EFI} folders on 1st and the porteus folder on 2nd, which would enable folder persistence, without the need for a .dat container.. (at least, as an option)
while i'm thinking of options, one for no persistence might be nice too...
another one: option to download a browser module from the mirror/modules folder and place it in porteus/modules

[robang74]

add a "sanity check", ie. check for the necessary utilities on the host system, say, with which

The `set -e` is there also for a reason. Checking that all stuff is in place is the LAST thing to do, because until it's done things might get in or out. Plus, using the BSD 3-clauses as license, rather than a "GPLv2 or later" I am impliciting claiming that "what you get is AS-IS". Finally, when I am saying "it uses bashism but in the long run I want it running with busybox a/sh", I am suggesting that a static linked parser in an AppImage-like binary is the way to port it to a broader set of systems. In this case there is no "sanity check" to do because everything is included into a single executable binary.

i notice you include the keymap cheatcode, this can be used in the /porteus/porteus-v5.0-x86_64.cfg file, which would make it work across all boot entries.
same for timezone= cheat, which might be nice to include as an option.

First of all /porteus/porteus-v5.0-x86_64.cfg file is a nice way to go but it is porteus-specific while a kernel argument works for every Linux system despite its application might vary depending from the bootloader to the bootloader. So, no - I did not read the porteous documentation neither I am gonna to knee the porteus way to do - for me porteus is just another Linux system and that's all. The same for the bootloader but one piece at time, so the boot loader being lower-level, it is still ruling as long as it serves its purposes.

Secondly, in the moment that I claimed that future improvements are gonna towards the direction of wget -O- | dd of=/dev/usb, boot in RAMFS and do the magic, it means that the only thing users need in the first place is having their keyboard immediately working. How? Well, wget -O- is just for bragging but #kmap=xx can be found into an ISO file and changed, as long as checksum, if any, is matched at run-time. Anyway, this is mainly a shortcoming of porteus but I am not gonna tell porteus developing team what or how to do their stuff.

So, the users download, write, and boot in RAMFS. At this point s/he can do all the configurations s/he likes and then use the 3rd script I am gonna to write to create on another USB or in the same USB from which s/he boot the system using my approach to create a persistence and to save its configuration into the persistence. Hence:

1 - download
2 - write
3 - boot
4 - config
5 - create

There is more, as you can imagine but for the generic porteus user, this is a nice way to go. So, if the porteus team wish to integrate the script and provide a way to change the keyboard on the fly (aka download ISO, check its SHA256 and altering the kmap before booting) then the generic user will magically find him/herself in the position to customise its porteous and save it having a living system without the need to change by hands config files and test it, fail, retry. The system config and creates itself. Here again, no sanity check is needed.

question: am i correct that two partitions are created 1st. fat32, 2nd. ext4 ?
if that's the case, it's possible to place the {boot,EFI} folders on 1st and the porteus folder on 2nd, which would enable folder persistence, without the need for a .dat container.. (at least, as an option)

Everything is possible, but then you should ask the porteus’ dev team why they did not do it in such a way as per default way to go. Again, I am not to teach them anything. I take what they did and I provide myself a reasonable way to customise and use it. Probably, they did this in their installation tools. Which again, I did not consider but once I boot into RAMFS mode, it might be THAT way, the way the imagined to go to create an installation. Because as you can imagine when a system runs on an ext4 while the boot is into VFAT32, that it is not anymore a live but a fully installed system.

Guess what? A full installed system is a thing. A pxe bootable system running on a single chunk of data - that can be sync by ethernet in real-time - is completely another thing. By the way, we do not sync stuff in real-time that has a journal filesystem running on it. Which is another reason because I disabled it when creating the changes.dat. Which is something that can be somewhere else and not necessarily on a USB.

Put all this together: a pxe boot (or a bootable usb), configure on RAMFS live, deploy, and network shared persistence. What is that? A thin client or if you prefer to have a broader vision, a Chrome OS alternative when Cloud as SaaS is provided instead of in-home administered Ethernet.

another one: option to download a browser module from the mirror/modules folder and place it in porteus/modules

Which is part of the customization in RAMFS. Why should I have to make a lot of choices for the user? Let him/her boot in RAMFS, customise his/her system and give the chance to create a bootable USB stick to run it out-of-their-pockets. Which is probably - exactly - what the porteus DEV team did in providing an installation GUI into porteus itself but I did not even care to give it a try.

--HUMOR ON--
Why the hell Roberto? Because a sysadm is an bastard operator from the Hell, or - in a more "professional" way - I do not care AND I SHOULD not care about porteous DEV team decision or schedule to deliver - I am in charge, as person as sysadm, to grant the service and to grant that the service will be Unix/Posix compliant whatever the DEV team does, and whatever “punishment” I have to enjoy to distribute among the not-root users in order to keep the system working.
--HUMOR OFF--

Does it sound crazy? Imagine the CTO/CFO as not-root users and you have the level of madness served. Hi Bill (Gates), I am the sysadm in charge of system root and I have "good" news for you! So, this is the reason because of BOFH (bastard operator from Hell).

Best regards, R-

POST SCRIPTUM

Before heating up, here Gemini opinion about my answer

-> https://g.co/gemini/share/a53246e2c198

[ncmprhnsbl]

Best regards, R-

lol, ok