[Solved] dir2xzm with a top level directory

Technical issues/questions of an intermediate or advanced nature.
jssouza
Legendary
Legendary
Posts: 1165
Joined: 09 Jul 2015, 14:17
Distribution: Porteus x86 arm

[Solved] dir2xzm with a top level directory

Post#1 by jssouza » 26 Aug 2016, 09:48

Hello,

I always used the command

Code: Select all

dir2xzm <directory> <module.xzm>
But today, I noticed that dir2xzm creates a top level directory if the <directory> starts with the names of the top level file system directories. (Saw it also implemented that way in the dir2xzm script).

I have a directory called libunity (with sub directories usr and var), now if I call

Code: Select all

dir2xzm libunity libunity.xzm
The squashfs xzm contains libunity as the top level directory and not usr and var. The solution ofcourse was for me to go into the libunity directoy and call

Code: Select all

dir2xzm . ../libunity.xzm
or rename the source directory to some other name that does not begin with the names of the top level directories I guess. But I would have to keep in mind this special feature of the script while creating modules.

Is there any specific reason for this feature in the dir2xzm script?

Thanks.
Last edited by jssouza on 28 Aug 2016, 15:02, edited 1 time in total.

User avatar
wread
Module Guard
Module Guard
Posts: 1255
Joined: 09 Jan 2011, 18:48
Distribution: Porteus v5.0-kde-64 bits
Location: Santo Domingo
Contact:

Re: dir2xzm with a top level directory

Post#2 by wread » 26 Aug 2016, 11:33

I don't understand what you mean.....I do build modules almost every day and have not experienced the behavior you describe :shock:
Porteus is proud of the FASTEST KDE ever made.....(take akonadi, nepomuk and soprano out and you will have a decent OS).
The Porteus Community never sleeps!

jssouza
Legendary
Legendary
Posts: 1165
Joined: 09 Jul 2015, 14:17
Distribution: Porteus x86 arm

Re: dir2xzm with a top level directory

Post#3 by jssouza » 26 Aug 2016, 11:54

Hi wread,

I give you an example:

Code: Select all

# mkdir -p binder/usr binder/var
# touch binder/usr/file1.txt
# touch binder/var/file2.txt
So the contents of the binder directory is:

Code: Select all

[root:~]# ls -R binder
binder:
usr  var

binder/usr:
file1.txt

binder/var:
file2.txt
[root:~]# 
Now, I duplicate the binder directory as:

Code: Select all

[root:~]# cp -a binder grinder
Now, if I create modules from these 2 directories:

Code: Select all

dir2xzm binder binder.xzm
dir2xzm grinder grinder.xzm

This is the content of the 2 modules:

Code: Select all

[root:~]# unsquashfs -l binder.xzm 
Parallel unsquashfs: Using 8 processors
2 inodes (0 blocks) to write

squashfs-root
squashfs-root/binder
squashfs-root/binder/usr
squashfs-root/binder/usr/file1.txt
squashfs-root/binder/var
squashfs-root/binder/var/file2.txt
[root:~]# unsquashfs -l grinder.xzm 
Parallel unsquashfs: Using 8 processors
2 inodes (0 blocks) to write

squashfs-root
squashfs-root/usr
squashfs-root/usr/file1.txt
squashfs-root/var
squashfs-root/var/file2.txt
You can see that the top level directory for binder.xzm is binder (and then usr and var as subdirectories), while for the grinder.xzm there is no top level grinder directory. This is because, "binder" has the substring "bin", a top level filesystem directory in the beginning.
So, I guess this should also be the same for directories like "variable*", "libreoffice*" etc.

User avatar
wread
Module Guard
Module Guard
Posts: 1255
Joined: 09 Jan 2011, 18:48
Distribution: Porteus v5.0-kde-64 bits
Location: Santo Domingo
Contact:

Re: dir2xzm with a top level directory

Post#4 by wread » 26 Aug 2016, 15:21

Hi jssouza
My accounting looks somewhat different to yours: When I check the contents of the 2 modules, I get

Code: Select all

root@porteus:~# lsxzm binder.xzm
Parallel unsquashfs: Using 2 processors
2 inodes (0 blocks) to write


/usr
/usr/file1.txt
/var
/var/file2.txt
root@porteus:~# lsxzm grinder.xzm
Parallel unsquashfs: Using 2 processors
2 inodes (0 blocks) to write


/usr
/usr/file1.txt
/var
/var/file2.txt
root@porteus:~# 
This is Porteus! :good:
Porteus is proud of the FASTEST KDE ever made.....(take akonadi, nepomuk and soprano out and you will have a decent OS).
The Porteus Community never sleeps!

jssouza
Legendary
Legendary
Posts: 1165
Joined: 09 Jul 2015, 14:17
Distribution: Porteus x86 arm

Re: dir2xzm with a top level directory

Post#5 by jssouza » 26 Aug 2016, 15:56

Hmm.. Not for me...

Here is my complete output again with the lsxzm script:

Code: Select all

[root:~]# mkdir -p binder/usr binder/var
[root:~]# touch binder/usr/file1.txt
[root:~]# touch binder/var/file2.txt
[root:~]# dir2xzm binder/ binder.xzm
binder/
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on binder.xzm, block size 262144.


Exportable Squashfs 4.0 filesystem, xz compressed, data block size 262144
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 0.43 Kbytes (0.00 Mbytes)
	91.21% of uncompressed filesystem size (0.47 Kbytes)
Inode table size 138 bytes (0.13 Kbytes)
	71.13% of uncompressed inode table size (194 bytes)
Directory table size 120 bytes (0.12 Kbytes)
	100.00% of uncompressed directory table size (120 bytes)
Number of duplicate files found 2
Number of inodes 6
Number of files 2
Number of fragments 0
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 4
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
[root:~]# lsxzm binder.xzm 
Parallel unsquashfs: Using 4 processors
2 inodes (0 blocks) to write


/binder
/binder/usr
/binder/usr/file1.txt
/binder/var
/binder/var/file2.txt
[root:~]# 
Could it be that you and I are using two different versions of the porteus scripts?

You do not get a top level "binder" directory even if you activate this xzm?

Code: Select all

[root:~]# activate binder.xzm 
Quiet mode: off
Updating shared library links:  /sbin/ldconfig
[root:~]# ls /mnt/live/memory/images/binder.xzm/
binder
[root:~]# ls -R /mnt/live/memory/images/binder.xzm/
/mnt/live/memory/images/binder.xzm/:
binder

/mnt/live/memory/images/binder.xzm/binder:
usr  var

/mnt/live/memory/images/binder.xzm/binder/usr:
file1.txt

/mnt/live/memory/images/binder.xzm/binder/var:
file2.txt
[root:~]# ls /
bin     dev  home  lib64  mnt  proc  run   srv  tmp  var
binder  etc  lib   media  opt  root  sbin  sys  usr
[root:~]# 

User avatar
brokenman
Site Admin
Site Admin
Posts: 6105
Joined: 27 Dec 2010, 03:50
Distribution: Porteus v4 all desktops
Location: Brazil

Re: dir2xzm with a top level directory

Post#6 by brokenman » 26 Aug 2016, 17:15

I also verify that it works ok for me (rc4) after following your procedure.

Code: Select all

root@porteus:/tmp# ls -R squashfs-root/
squashfs-root/:
dir1/  dir2/

squashfs-root/dir1:
file1.txt

squashfs-root/dir2:
file2.txt
Can you please post the md5sums for dir2xzm?
How do i become super user?
Wear your underpants on the outside and put on a cape.

jssouza
Legendary
Legendary
Posts: 1165
Joined: 09 Jul 2015, 14:17
Distribution: Porteus x86 arm

Re: dir2xzm with a top level directory

Post#7 by jssouza » 26 Aug 2016, 19:37

Hi brokenman,

Here you go:

Code: Select all

[root:~]# md5sum /opt/porteus-scripts/dir2xzm 
c141c4893ca5ac759cf693b3b8652e60  /opt/porteus-scripts/dir2xzm
Thank you for looking into this.

User avatar
brokenman
Site Admin
Site Admin
Posts: 6105
Joined: 27 Dec 2010, 03:50
Distribution: Porteus v4 all desktops
Location: Brazil

Re: dir2xzm with a top level directory

Post#8 by brokenman » 27 Aug 2016, 01:41

Please open /opt/porteus-scripts/dir2xzm and navigate to line: 44

Change:
# Check if we need to create root for the module:
echo $1 | egrep '^bin|^etc|^home|^lib|^opt|^root|^sbin|^srv|^usr|^tmp|^var' && keep='-keep-as-directory'

to

# Check if we need to create root for the module:
echo $1 | egrep -w '^bin|^etc|^home|^lib|^opt|^root|^sbin|^srv|^usr|^tmp|^var' && keep='-keep-as-directory'

Thanks for the bug report.
How do i become super user?
Wear your underpants on the outside and put on a cape.

jssouza
Legendary
Legendary
Posts: 1165
Joined: 09 Jul 2015, 14:17
Distribution: Porteus x86 arm

Re: dir2xzm with a top level directory

Post#9 by jssouza » 27 Aug 2016, 08:03

Thank you for the fix brokenman, it now works fine.

I also read the egrep manual for the -w option and understood what it now does in the script.

Just out of curiosity, why was it not reproducible with you and wread?

User avatar
wread
Module Guard
Module Guard
Posts: 1255
Joined: 09 Jan 2011, 18:48
Distribution: Porteus v5.0-kde-64 bits
Location: Santo Domingo
Contact:

Re: dir2xzm with a top level directory

Post#10 by wread » 27 Aug 2016, 12:50

of course it was reproducible by me, but I used the method I normally use, that worked ok, namely using lsxzm...
Porteus is proud of the FASTEST KDE ever made.....(take akonadi, nepomuk and soprano out and you will have a decent OS).
The Porteus Community never sleeps!

User avatar
brokenman
Site Admin
Site Admin
Posts: 6105
Joined: 27 Dec 2010, 03:50
Distribution: Porteus v4 all desktops
Location: Brazil

Re: dir2xzm with a top level directory

Post#11 by brokenman » 27 Aug 2016, 22:09

Just out of curiosity, why was it not reproducible with you and wread?
Because I was an idiot and didn't thoroughly read your post and follow your instructions. After the md5sum I reread and I saw the name 'binder' and the penny dropped.
How do i become super user?
Wear your underpants on the outside and put on a cape.

jssouza
Legendary
Legendary
Posts: 1165
Joined: 09 Jul 2015, 14:17
Distribution: Porteus x86 arm

Re: [Solved] dir2xzm with a top level directory

Post#12 by jssouza » 28 Aug 2016, 15:01

Ah ok.

Well, looking at how you have developed Porteus, for me you are sheer genius.

Thanks again!

Jack
Contributor
Contributor
Posts: 1857
Joined: 09 Aug 2013, 14:25
Distribution: Porteus and Nemesis
Location: USA

Re: dir2xzm with a top level directory

Post#13 by Jack » 30 Aug 2016, 02:43

brokenman wrote:Please open /opt/porteus-scripts/dir2xzm and navigate to line: 44

Change:
# Check if we need to create root for the module:
echo $1 | egrep '^bin|^etc|^home|^lib|^opt|^root|^sbin|^srv|^usr|^tmp|^var' && keep='-keep-as-directory'

to

# Check if we need to create root for the module:
echo $1 | egrep -w '^bin|^etc|^home|^lib|^opt|^root|^sbin|^srv|^usr|^tmp|^var' && keep='-keep-as-directory'

Thanks for the bug report.
Should I fix it or should I wait for the next release? I do use it that why I'm asking.
I just like Slackware because I think it teach you about Linux to build packages where Ubuntu is like Windows you just install programs you want.

jssouza
Legendary
Legendary
Posts: 1165
Joined: 09 Jul 2015, 14:17
Distribution: Porteus x86 arm

Re: [Solved] dir2xzm with a top level directory

Post#14 by jssouza » 30 Aug 2016, 11:32

Hi Jack,

I downloaded http://ftp.vim.org/ftp/os/Linux/distr/p ... x86_64.iso and looked in the 001-core.xzm/opt/porteus-scripts/dir2xzm and saw that the fix is present.

However, I would wait until the rc5 release is officially announced and then download the iso.

Jack
Contributor
Contributor
Posts: 1857
Joined: 09 Aug 2013, 14:25
Distribution: Porteus and Nemesis
Location: USA

Re: [Solved] dir2xzm with a top level directory

Post#15 by Jack » 30 Aug 2016, 20:01

jssouza wrote:Hi Jack,

I downloaded http://ftp.vim.org/ftp/os/Linux/distr/p ... x86_64.iso and looked in the 001-core.xzm/opt/porteus-scripts/dir2xzm and saw that the fix is present.

However, I would wait until the rc5 release is officially announced and then download the iso.
Thanks I already downloaded it early today but at 7:00AM had to go to VA doctor and just got home. I'm going in a few minute.
I just like Slackware because I think it teach you about Linux to build packages where Ubuntu is like Windows you just install programs you want.

Post Reply