Porteus v1.2 rc2 x86_64 is ready for testing!

[jcuk]

>[..]check out my previous comments in this thread about Network Manager and play with this app for a while

Cant find it! Looked all over the LXDE and KDE Desktop/menus. Root and Guest. How do I invoke it? Where is it?

[fanthom]

@cttan

I have put this in my post-startup script to copy the files when Porteus is up and running:-
cp -ar /mnt/sdb1/etc/NetworkManager/. /etc/NetworkManager/.

is /mnt/sdb1 on a linux fs or FAT? i'm guessing that maybe 777 permissions are not letting NM to start.

anyway - please create /porteus/rootcopy/etc/rc.d/rc.S/S99network-manager with following content:

Code: Select all

cp -ar /mnt/sdb1/etc/NetworkManager/. /etc/NetworkManager/.

run 'chmod +x /porteus/rootcopy/etc/rc.d/rc.S/S99network-manager' and reboot porteus. should work.
you could also create module containing NM settings and use load/noload cheats.

@jcuk
1) right mouse click on NM icon from the tray -> Edit Connections
2) kmenu/lxdemenu -> Preferences/System Settings -> Network Connections

[sams]

I've been doing a bunch of testing of the LUKS encrypted container, with bad results. The luks mount in rc.S has some scary messages, the mount actually seems to work but though I see the decrypted contents of the luks container, the filesystem reports the free space of the container's filesystem rather than the free space of the container. When I see these symptoms, I can't completely unwind the encrypted mount, the cryptsetup luksClose may fail, or it may be impossible to unmount, or to delete the loop device. Sorry I don't have exact error messages, just specifying an encrypted container for a magic folder mount has corrupted a couple of containers, which slows down testing. Also I don't get clean unmounts of the filesystem and or the luks container.

What is also a bit odd is that I also have my own scripts to mount and unmount a container for home directories. My code is substantially similar to fanthom's code in rc.S and rc.K. My own code does a more careful job of unwinding all the resources of the luks mount for all kinds of errors at every step. In particular it's a bit tricky to delete /dev/loop* files, you can _successfully_ rm the file, and then test it and it's still there. Eventually if you sync and wait you can really delete it, but the rm succeeds whether it makes the file go away or not (!). Also, my code is run from rc.local and rc.local_shutdown, it doesn't seem like this should make a difference, but it does pick up a way different loop device for running later (which is odd).

So when I use my own code to mount and unmount the luks container, either from the rc scripts or later, the filesystem works perfectly and is always left in a clean state. So far I don't have similar luck with the magic folders code, not sure why (doesn't make a lot of sense). I'll try and post my mount and unmount code tomorrow night if your experience with encrypted containers is different.

Posted after 1 day 13 hours 9 minutes 14 seconds:
I'm appending my scripts to bring up and down the luks container for my home directory. I made these reversible so you can endable and disable the container repeately without leaking resources. Also it mounts the container in a manner that I believe should be compatible with the magic folders implementation, though as I said I haven't had good luck with delaying clean home dir unmounting and letting rc.K magic folder code clean up.

I mount the container on a directory I create in /tmp, which in my porteus is a tmpfs mount. It's possible that the magic folder code has order dependencies that don't like my configuration. I have my workaround anyway by mouting in rc.local and unmounting in rc.local_shutdown. I apologize if I am reporting bugs that don't manifest in the stock configuration. Anyway the rationale behind a home directory on a tmpfs at /tmp/containerMountPoint/userDir is that the user only has a directory and persistence when everything is copacetic, otherwise his place is pretty clearly nonexistent or ephemeral.

/usr/local/bin/mountcrybaby:

Code: Select all

#!/bin/sh

if [ -e /tmp/containerMountPoint/crybaby ] ; then
    echo "~crybaby already mounted"
    exit
fi

# consider encrypting swap
swapoff -a
if [ $? -ne 0 ]; then
    echo "swapoff failed, you could leak clear data"
fi


if [ ! -e /tmp/containerMountPoint ] ; then
    mkdir /tmp/containerMountPoint
    chown crybaby:myGroup /tmp/containerMountPoint
    chmod 750 /tmp/containerMountPoint
fi

CONTAINERDAT=`ls /mnt/*/porteus/container.dat 2> /dev/null | tail -n 1`
if [ "x$CONTAINERDAT" == "x" ]; then
    echo "container.dat not found"
    exit 2
fi

LOOPNUM=`/usr/local/bin/makeloop`
LOOPDEV=/dev/loop$LOOPNUM

losetup $LOOPDEV $CONTAINERDAT
LOSETUP_RESULT=$?
if [ $LOSETUP_RESULT -ne 0 ]; then
    echo "losetup $LOOPDEV $CONTAINERDAT failed: $LOSETUP_RESULT"
    rm $LOOPDEV ; sync
    if [ -e $LOOPDEV ] ; then
        #echo "wtf $LOOPDEV still here"
        sleep 2 ; rm $LOOPDEV ; sync
        if [ -e $LOOPDEV ] ; then
	    echo "again, wtf $LOOPDEV wont go away"
        fi
    fi
    exit 1
fi

cryptsetup luksOpen $LOOPDEV magic$LOOPNUM
if [ $? -ne 0 ]; then
    echo "cryptsetup failed"
    losetup -d $LOOPDEV

    rm $LOOPDEV ; sync
    if [ -e $LOOPDEV ] ; then
        #echo "2wtf $LOOPDEV still here"
        sleep 2 ; rm $LOOPDEV ; sync
        if [ -e $LOOPDEV ] ; then
	    echo "2again, wtf $LOOPDEV wont go away"
        fi
    fi
    exit 1
fi

/sbin/fsck /dev/mapper/magic$LOOPNUM
mount -o noatime /dev/mapper/magic$LOOPNUM /tmp/containerMountPoint
MOUNT_RESULT=$?

if [ $MOUNT_RESULT -ne 0 ]; then
    echo "mount of ~crybaby failed"
    cryptsetup luksClose magic$LOOPNUM
    losetup -d $LOOPDEV
    rm $LOOPDEV ; sync
    if [ -e $LOOPDEV ] ; then
        #echo "3wtf $LOOPDEV still here"
        sleep 2 ; rm $LOOPDEV ; sync
        if [ -e $LOOPDEV ] ; then
	    echo "3again, wtf $LOOPDEV wont go away"
        fi
    fi
    exit $MOUNT_RESULT
fi

echo "~crybaby successfully mounted"
exit 

/usr/local/bin/umountcrybaby:

Code: Select all

#!/bin/sh

if [ ! -e /tmp/containerMountPoint/crybaby ] ; then
    #echo "umountcrybaby: nothing to do"
    exit
fi

sync ; sync
umount /tmp/containerMountPoint
UMOUNT_RESULT=$?

LOOPDEV=`losetup -a | grep "container\.dat" | tail -n 1 | awk -F : '{print$1}'`

if [ $UMOUNT_RESULT -eq 0 ]; then
    echo "umount /tmp/containerMountPoint succuss"
    sync

    LOOPNUM=`losetup -a | grep "container\.dat" | tail -n 1 | awk -F : '{print$1}' | sed s@/dev/loop@@`
    cryptsetup luksClose magic$LOOPNUM
    CRYPTSETUP_RESULT=$?
    if [ $CRYPTSETUP_RESULT -ne 0 ]; then
	echo "cryptsetup luksClose failed"
	exit 1
    fi
else
    echo "umount /tmp/containerMountPoint failed"
    exit 1
fi

if [ "$LOOPDEV" != "" ]; then
    sync
    losetup -d $LOOPDEV
    if [ $? -ne 0 ]; then
	echo "losetup -d $LOOPDEV failed"
	exit 1
    fi

    #echo "really deleting $LOOPDEV I hope"
    rm $LOOPDEV ; sync
    if [ -e $LOOPDEV ] ; then
        #echo "wtf $LOOPDEV still here"
        sleep 2 ; rm $LOOPDEV ; sync
        if [ -e $LOOPDEV ] ; then
            echo "again, wtf $LOOPDEV wont go away"
            sleep 2 ; rm $LOOPDEV ; sync
        fi
        if [ -e $LOOPDEV ] ; then
            exit 1
        fi
    fi
fi

rmdir /tmp/containerMountPoint
if [ $? -ne 0 ]; then
    echo "rmdir /tmp/containerMountPoint failed"
fi

sync

/usr/local/bin/makeloop:

Code: Select all

#!/bin/bash
# Script to make a new loop device

x=`ls -1 /dev/loop* | awk -F/ '{print$3}' | tr -d [:alpha:] | sort -n | tail -n1`
lp=$(($x+1))

#Create a new loop
mknod /dev/loop$lp b 7 $lp

echo $lp

ps I apologise for the ugly bash code!

[fanthom]

@sams

The luks mount in rc.S has some scary messages

i think you are refering to a padlock_sha.ko warning which appears when cryptsetup is opening the container and tries to initialize HW accelerators present on some rare VIA and geode CPU's.
this warning appears only because our logging facility (syslog) is not started yet as it's done in rc.M.
i'm going to move magic folders binding to rc.M and start it after syslog (syslog will be restarted if binding is done on /var* folder)

the filesystem reports the free space of the container's filesystem rather than the free space of the container.

not sure what you mean by that...
df -h will always report filesystem size even if partition is bigger (like the case when user wants to resize it on the fly).

I can't completely unwind the encrypted mount, the cryptsetup luksClose may fail, or it may be impossible to unmount, or to delete the loop device.

i'm not experiencing these issues. maybe it's your custom scripts fault or something...

btw: containers are closed in rc.6 and not rc.K.
rc.K is used only when you are going into single user mode while rc.6 is user for reboots and shutdowns - maybe that's the reason of your failures?
also - no need to delete loop device - just detach it ('umount -d /folder' command)

i'll upload latest rc.* scripts when i return home so you can test stock porteus configuration for 1.2 final.
as i wrote before - no problems here with stock config.

Posted after 6 hours 50 minutes:
here they are:
http://www.4shared.com/file/uBfi4aE4/rc ... devel.html

[sams]

i think you are refering to a padlock_sha.ko (...) syslog is not started yet

Yes, that's it. Thanks for info.

the filesystem reports the free space of the container's filesystem rather than the free space of the container.

Here I misspoke, what I am seeing is the free space of the file system of the mount point (!). In the following, my container is a 630MB luks container residing on flash drive (sdc1), with luks mount point on the tmpfs (which can grow to half of RAM, 8GB).

If I put this in my folders.cfg: (and precreate the mountpoint)

Code: Select all

/mnt/*/porteus/container.dat /tmp/containerMountPoint

Here's the resulting filesystem result:

Code: Select all

$ df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/sdc1            7.4G  1.7G  5.4G  24% /mnt/sdc1
/dev/mapper/magic13  7.8G  144M  7.7G   2% /tmp/containerMountPoint
tmpfs                7.8G  144M  7.7G   2% /tmp

Ack! The device mapper is decryting my 630 MB container, but the free space is that of the tmpfs that is the container's mount point.

The problem could be an interaction with my code in rc.local to move /tmp to a tmpfs:

Code: Select all

### make /tmp really temporary on a tmpfs
### make the mount point
mkdir /mnt/tmp

### put a ram filesystem on the mount
mount -t tmpfs -o noatime tmpfs /mnt/tmp

### copy the old tmp contents so we retain present state
( cd /tmp ; tar cf - . ) | ( cd /mnt/tmp ; tar xf -)
( cd /var/tmp ; tar cf - . ) | ( cd /mnt/tmp ; tar xf -)

mount --move /mnt/tmp /tmp
rmdir /mnt/tmp

### bind overmount to replace original /var/tmp
mount --bind /tmp /var/tmp

This hack is conceptually a bit ugly, there could be open file descriptors in the old /tmp dir, but this doesn't cause any problems. Is /tmp on a tmpfs in RAM on a live system the best choice? I'm not prepared to make the argument that it should be the system default (I think porteus & slax have excellent default settings) but in my setups its all advantages. It's fast, it doesn't wear the flash drive, and it's really temporary.

So I think the only code that's particular to my setup is where I move /tmp to a tmfs. Other than the mount point, my configuration looks like standard usage of the magic folders code.

If I don't use the magic folders code, but instead mount the container with the 'mountcrybaby' script above (either invoked from rc.local or later by a user) the container decrypts to show the same directory structure, but this time the filesystem free space is correct:

Code: Select all

$ df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/sdc1            7.4G  1.7G  5.4G  24% /mnt/sdc1
tmpfs                7.8G  102M  7.7G   2% /tmp
/dev/mapper/magic20  631M   44M  581M   7% /tmp/containerMountPoint

So in either case the mount point shows my decrypted container contents, but if the mounting is handled by magic folders I show the wrong free space and I can't unmount cleanly. If I use my own scripts included above and invoked from the rc.locals or a shell, everything works perfectly and stays clean. The other thing that I'm curious about is the device number. For me, magic folders mounts the container at loop13, if I run my 'mountcrybaby' script from rc.local the device is loop20. Since you can see from my 'umountcrybaby' script loop devices are bouncy and hard to delete I wonder if these are getting leaked? Sorry I don't know how to check kernel associations with loop devices.

btw: containers are closed in rc.6 and not rc.K.

Thanks, my bad. I blame Slurpee and brain freeze.

also - no need to delete loop device - just detach it ('umount -d /folder' command)

I will investigate & learn!

I'm suspecting that my problem might be an interaction between my tmpfs desires and abuses and the porteus shutdown order. It's possible that relying on cleanup in rc.6 in this scenario precludes clean shutdown, and then the device mapper is scrogged on remount? It wasn't clear from my reading of the shutdown code that this ought to be problematic, but it has been a no go for me. But I'm going to stick with the tmpfs, you don't want to put an encrypted home directory in a location where it might exist and get archived in the clear if the user errored out of the luksOpen.

I will try your new scripts shortly. Thanks for all.

cheers!

[fanthom]

hi sams,

i guess the problem is in the mounting order of tmpfs and your encrypted container.
when tmpfs is mounted on top of container then df -h gets confused and display incorrect size. wrong order could be also to blame for dirty shutdowns.

please make sure that your rc.locl script mounts tmpfs in /tmp first and then mount container in /tmp/containerMountPoint.
rc.local_shutdown should unmount /tmp/containerMountPoint and then /tmp.

in latest rc scripts posted above, magic folders part was moved to rc.M so you have to mount tmpfs in /tmp through sysvinit scripts for runlevel S and all should be ok.

[sams]

Thanks for help with this fanthom, I think I get it now, I was surprised that I seemed to create a shutdown order problem, now I'm only surprised that I've been a bit slow...

I've been using rc2 extensively without trouble, here's the only other nits I've seen:

Network manager never remembers the setting to stop notifiying me that it found my wired network on startup.

I experienced a bug where as user 'guest' I could not successfully edit menus in kmenuedit. Probably relevant is that I'm using a save.dat container. The root of the problem was that /home/guest/.local/share/applications/ could not be entered by guest. Permissions on the directories looked perfect:

Code: Select all

guest@porteus:~/.local/share
$ ls -la
drwxr-xr-x 5 guest guest  37 May 31 08:46 .
drwxr-xr-x 4 guest guest  18 Apr 11  2011 ..
drwx------ 2 guest guest 131 May 31 08:52 applications
drwx------ 4 guest guest  29 Mar 12 19:00 Trash

So guest could cd to ~/.local/share, but not to ~/.local/share/applications. Root could cd into the applications directory just fine. I was going to recreate this directory as a test, but just renaming restored sanity; Once it was renamed 'applications.new' guest could cd just fine, and guest could continue to enter and use the directory when I named it back to 'applications'. To me this looked like an aufs error, I will see if it is reproducible. Anyway once I renamed the directory and named it back, kmenuedit is able to create .desktop files and the menu editor works fine. Nothing about permissions indicated why there would be an error, and permissions didn't change after the problem was fixed.

Later Edit: this problem doesn't occur on stock porteus. It might yet be an aufs bug (or not), but certainly relevant is that the directory appears in 003-lxde.xzm/home/guest/.local/share/applications and my guest has a different UID. Using stock porteus I decompress my own environment module and examine the relevant directory:

Code: Select all

root@porteus:/tmp/xxx/rrr/home/guest/.local/share/applications# ls -la
total 4
drwx------ 2 31337 31337  60 Sep 15  2011 ./
drwxr-xr-x 4 31337 31337  80 Mar 13 05:00 ../
-rw-r--r-- 1 31337 31337 139 Sep 15  2011 mimeapps.list

In my porteus, guest has UID & GID of 31337 yet fails to cd to this applications directory unless I seem to have modified the 'applications' dir in the save.dat. I will have to think about the full conditions that reproduce what feels like a bug, not sure if this is my problem or a bug.

I haven't looked at the new startup scripts yet, 4shared site needs a sandbox as it seems to be a bottomless pit of cross-site scripting errors before requiring a password. Will navigate later...

cheers

Posted after 1 hour 58 minutes 37 seconds:
(Another later edit)

With further testing, all the problems in this post are the same problem, specific to my guest account with its modifed UID/GID. Most directories work fine, but there are were 2 directories (also ~guest/.gconf which broked networkManager) that all have this odd behaviour, they look like fine permissions but don't let the new UID in until root renames the directory to the same name. This breaks some guest configuration until I retouch the directories. I don't know which I hope for less, an obscure aufs bug or another obscure interaction with my rc.local code. Ack! I will figure it out, sorry for the noise!

[sams]

Another nit: There is no linux executable for syslinux in /boot/syslinux, so you can't boot the iso and use it to make a bootable FAT drive. In the past, I have used Slax to unpack the iso, and I have run Porteus extlinux from Slax to make a bootable flash drive; making this 32 bit executables was an appreciated touch. It looks like Porteus developers anticipate doing something similar from Windows? Anyway, seems like this should work from a Porteus iso boot.

cheers!

[fanthom]

There is no linux executable for syslinux in /boot/syslinux, so you can't boot the iso and use it to make a bootable FAT drive.

a) we dont need syslinux as extlinux supports FAT+ext2/3/4 (and also btrfs but booting from it is not supported in porteus right now)
b) you should be able to boot from porteus iso and make bootable FAT drive by either using 'Porteus installer' or copying data manually to destination partition and running /mnt/sdXY/boot/lin_start_here.sh script.

please try second method and show me the error which you get.

[sams]

I did not know extlinux now supports FAT, again I apologise and thank you for explanation. I have not experienced any problem because so far I have only used ext2, and I just install manually (it's just 2 lines, I hadn't even noticed the install scripts)

On the bright side I haven't actually had any difficulties doing everything I need so I hadn't looked for any docs. Now that I'm looking around I see there is plenty and I embarass myself with old habits. As wife says I must look with eyes and not mouth.

best regards

[sams]

Hi again, I have a condition with rc2 that I doesn't quite make sense, maybe someone can try to duplicate to see if this is a bug. I have been running Porteus often with cheatcodes "copy2ram noauto" ; this feels like a prety safe mode and I have all my USB ports free. This works perfectly from my flash drive (8 gigs, ext2), stock porteus with no additional modules. But I have tested 2 usb hard drives (>250 gigs, ext3) and the symptoms looks like some automounter dies. In the GUI, when I click on the USB notification panel to mount the drive, it says "Could not mount the following device:" followed by whatever device I try, nothing automounts at this point.

The drives fsck as clean and can be manually mounted no problem, also work fine on all other systems. This command doesn't hint at anything substantial in the logs:

# cd /var/log ; grep sdc *

From this the only interesting thing I see is:
porteus-livedbg:/dev/sdc1: LABEL="wd250" UUID="2bc6-bebe-bb0d-01234" SEC_TYPE="ext2" TYPE="ext3"

I can't explain why the big spinning disks exhibit different behaviour here for me than the usb flash, but I've tested it on 2 drives with the exact same image as the flash drive, this booting porteus in stock form with "copy2ram noauto" If I omit the "noauto" the GUI mounter works fine on all drives when I plug them in.

I can provide more info, but I'm not sure where to look, I don't find obvious error indications in /var/log.

cheers

ps I still haven't read any docs, if I am missing something documented you should ignore me...

[fanthom]

@sams

1) by default 'guest' has no permission to deal with devices listed in /etc/fstab (only root can do that) as these are considered as 'internal'. guest can mount/umount removable devices which are added/removed during guest session.
2) you can change that behavior with modifying default mount options by using 'mopt=' cheatcode - just add 'users' flag like here:

Code: Select all

copy2ram noauto mopt=users,noatime,nodiratime,suid,dev,exec

or whatever you want.

references:
/boot/docs/cheatcodes.txt
'man mount'

hope that helps.

[sams]

Thanks fanthom, with your tip I see the contents of fstab are at the root of my problem. If I boot from usb flash, the drive is seen as removable and not listed in fstab. If I boot from a USB hard drive, the drive reports not removable in /sys/block/*/removable, so the USB hard drive's partitions appear in fstab. Then if I try to mount in the GUI, any drive that maps to (in my case) sdc1 or sdc2 cannot be mounted by the non-root process of the USB toolbar mounter. Any media that I try to mount after the USB hard drive has been ejected fails because sdc* didn't get removed from fstab.

Same problem as mentioned here:

http://www.sysresccd.org/forums/viewtop ... f=5&t=3211
http://ubuntuforums.org/showthread.php?t=1359208

In these discussions, I didn't see that a good answer has been found. I don't know yet if all USB hard drives report the removable attribute in this tricky way:

GENHD_FL_REMOVABLE should:
* - be set for removable media with permanent block devices
* - be unset for removable block devices with permanent media

yuck

[brokenman]

Ouch. I check for removable devices often in scripts using this method. That's not good news.

[sams]

I use save.dat manager to make a file 1800 MB on my USB stick and it works fine. I make save.dat file 2600 MB and it fails to mount it (fails to touch ._test1 or something like that IIRC) I can point at problematic linuxrc line if it's not obvious (but source of problem is not obvious to me)

These tests were with xfs save.dat file. Also fails on big (8G) ext4. USB stick for porteus & save.dat is formatted ext2. I can 'mloop' mount the save.dat container fine from GUI.