[HOWTO] fix module activation - folder (or file) doesn't appear in live system

[Rava]

Under usual circumstances, the .wh* files as described here do their job as intended and there usually is never a need to delete any of these from your life system.


For certain reasons unrelated to this HOWTO I deleted the Basilisk's settings folder as guest via

Code: Select all

rm -r ~/.basilisk-dev/

, and loaded my basilisk settings folder module 986-basilisk-settings--2023-11-10.xzm - only to realize that the settings would not be applied and all files and folders from the 986-basilisk-settings--2023-11-10.xzm module would not appear in my life system - because the system created a white out file to "remember" that the folder /home/guest/.basilisk-dev/ got deleted.

Thanks to the helpful folks in the original thread in Intermediate/Advanced User Questions (link to that thread at the very bottom) I recalled the reason:
The way a live system works, and because all modules are read-only, there a special white out files that act as "this file or folder (and all sub-folders and sub-files of that very folder) should be hidden because the user deleted that file or folder" technique.

You can manually re-create that file or folder, but any such files from a later loaded module that sit in or "below" that folder path would still not appear in your life system, only the manually re-created file or folder would appear.
With "below" I mean this:
In my case, I deleted as guest ~/.basilisk-dev/ meaning the deleted folder was
/home/guest/.basilisk-dev/
and all and any files and folders "below" /home/guest/.basilisk-dev/ that are part of later activated module would still not appear in your life system:
e.g.

Code: Select all

/home/guest/.basilisk-dev/basilisk/
/home/guest/.basilisk-dev/basilisk/RaNdOm123.default/
/home/guest/.basilisk-dev/basilisk/profiles.ini

and all any any folders, files and sub-folders from /home/guest/.basilisk-dev/basilisk/RaNdOm123.default/ would not appear even when they are part of a later activated module.

You have to find the one singular responsible .wh* file and only delete that one singular .wh* file.

The location is always to be found in /mnt/live/memory/ but the precise path could be different, depending on if you use the changes cheat code - or not. I do not use the changes cheat code and in my case this command gave the the file responsible:

Code: Select all

find /mnt/live/memory/changes/home -name ".wh.*"

but when using the changes cheat code this might be your find command:

Code: Select all

find /mnt/live/memory/images/changes/home -name ".wh.*"

In my case, this was the responsible white out file:

Code: Select all

/mnt/live/memory/changes/home/guest/.basilisk-dev/.wh..wh..opq

You an identify it because it sits in the deleted folder that was /home/guest/.basilisk-dev/
There have been more hits for that search via the above find command, but I only deleted that one singular /mnt/live/memory/changes/home/guest/.basilisk-dev/.wh..wh..opq white out file to fix the issue and left all other white out files be.

Please take special care to not delete all .wh* files in your life system, only delete a single one when you encounter a similar issue as described here.

The original thread with some more background info can be found here: Module activation - folder doesn't appear in live system

[Ed_P]

Where do your .wh. files exist rava? In RAM or on your /porteus drive? And how many do you currently have? What do you do when they fill up your RAM or /porteus drive? Does deactivating a module cause .wh. files for each file in the module?

[Rava]

Where do your .wh. files exist rava? In RAM or on your /porteus drive? And how many do you currently have?

I RAM, why should they exist in my /porteus drive?
Currently I have these:

Code: Select all

root@rava:~# find /mnt/live/memory/changes/ -name ".wh.*"
/mnt/live/memory/changes/.wh.Porteus_modules_sdb4
/mnt/live/memory/changes/home/guest/.local/share/gvfs-metadata/.wh.home-c957bf62.log
/mnt/live/memory/changes/var/lib/pkgtools/packages/.wh.pango-1.50.6-x86_64-1-ncm
/mnt/live/memory/changes/.wh..wh.orph
/mnt/live/memory/changes/.wh..wh.plnk
/mnt/live/memory/changes/.wh..wh.aufs

and these are their sizes according to ls -o:

Code: Select all

root@rava:~# ls -o $(find /mnt/live/memory/changes/ -name ".wh.*")
-r--r--r-- 4 root  0 2023-11-10 07:17 /mnt/live/memory/changes/.wh..wh.aufs
-r--r--r-- 4 root  0 2023-11-10 07:17 /mnt/live/memory/changes/.wh.Porteus_modules_sdb4
-r--r--r-- 4 root  0 2023-11-10 07:17 /mnt/live/memory/changes/home/guest/.local/share/gvfs-metadata/.wh.home-c957bf62.log
-r--r--r-- 4 root  0 2023-11-10 07:17 /mnt/live/memory/changes/var/lib/pkgtools/packages/.wh.pango-1.50.6-x86_64-1-ncm

/mnt/live/memory/changes/.wh..wh.orph:
total 0

/mnt/live/memory/changes/.wh..wh.plnk:
total 0

As you can see, ".wh.*" files seems to be either empty files or empty folders.

What do you do when they fill up your RAM

Fat chance that empty files could ever fill up my RAM. Or any of my drives.

Let's ask stat about one such file:

Code: Select all

root@rava:~# stat /mnt/live/memory/changes/.wh..wh.aufs
  File: /mnt/live/memory/changes/.wh..wh.aufs
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 0,18	Inode: 5           Links: 4
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-11-10 07:17:19.823658407 +0100
Modify: 2023-11-10 07:17:19.823658407 +0100
Change: 2023-11-14 09:03:11.251333550 +0100
 Birth: -

You see the Size: 0 Blocks: 0 part if the initial line: Blocks: 0 means the file contents (which is zero) takes zero blocks.
One such file would take zero blocks of storage, and 1000 such files would also take zero blocks of storage. Aside from an entry in the directory they sit in.

Does deactivating a module cause .wh. files for each file in the module?

No idea, but that you can test for yourself. Now gotta go, RL is calling.

[Ed_P]

Fat chance that empty files could ever fill up my RAM. Or any of my drives.

Let's ask stat about one such file:

Code: Select all

root@rava:~# stat /mnt/live/memory/changes/.wh..wh.aufs
  File: /mnt/live/memory/changes/.wh..wh.aufs
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 0,18	Inode: 5           Links: 4
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-11-10 07:17:19.823658407 +0100
Modify: 2023-11-10 07:17:19.823658407 +0100
Change: 2023-11-14 09:03:11.251333550 +0100
 Birth: -

Rather than ask about a file that was deleted twice and in aufs what about;

Code: Select all

stat /mnt/live/memory/changes/.wh.*

[Rava]

Rather than ask about a file that was deleted twice and in aufs what about;

Code: Select all

stat /mnt/live/memory/changes/.wh.*

All have Blocks: Zero:

Code: Select all

root@rava:~# stat /mnt/live/memory/changes/.wh.*|grep Blocks
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
  Size: 40        	Blocks: 0          IO Block: 4096   directory
  Size: 40        	Blocks: 0          IO Block: 4096   directory
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
root@rava:~# 

Is that different for you that you use the changes cheat code?

[Ed_P]

The stats are about e same,

Code: Select all

guest@porteus:~$ stat /mnt/live/memory/changes/.wh.*|grep Blocks
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
  Size: 40        	Blocks: 0          IO Block: 4096   directory
  Size: 40        	Blocks: 0          IO Block: 4096   directory
guest@porteus:~$ 

Not sure what the size is measuring if the blocks are 0.

The concern I have with .wh. files is the length of the file names. For example:

Code: Select all

/mnt/live/memory/changes/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/https+++www.facebook.com/cache/morgue/.wh.103

Each character of that useless file's directory name represents a byte of RAM. And when you multiply that count by 850+ that's a fair chunk of RAM being wasted. And when they are saved to the save.dat file .... That's why I delete them.

[Rava]

Each character of that useless file's directory name represents a byte of RAM. And when you multiply that count by 850+ that's a fair chunk of RAM being wasted. And when they are saved to the save.dat file .... That's why I delete them.

That is still a minimal amount of RAM, just test it out: open a new tab in your terminal emulator.
Make a thorough snapshot of current RAM use prior opening one simple terminal tab, and one snapshot after opening that tab. Usually that will cost you more RAM than all such .wh* files put together, or at least one new tab in your terminal emulator equals all RAM of all all such .wh* files.
But still, it is a good idea to remove all and any .wh* files when creating a module, not just a changes module. Usually these only make sense in the live system.

Of course there could be a certain reason why you need to hide or delete a certain file or folder via a white out file that sits in one of your modules, but that would be a very rare and specific reason why that would make sense.

Added in 8 minutes 8 seconds:

Not sure what the size is measuring if the blocks are 0.

Code: Select all

guest@rava:~$ ln -s /mnt/sda1/invalid_folder /invalid_symlink
guest@rava:~$ stat /invalid_symlink
  File: /invalid_symlink -> /mnt/sda1/invalid_folder
  Size: 24        	Blocks: 0          IO Block: 4096   symbolic link
Device: 0,19	Inode: 68708       Links: 1
Access: (0777/lrwxrwxrwx)  Uid: ( 1000/   guest)   Gid: (  100/   users)
Access: 2023-11-16 09:34:02.836386978 +0100
Modify: 2023-11-16 09:34:02.836386978 +0100
Change: 2023-11-16 09:34:02.836386978 +0100
 Birth: -
guest@rava:~$ file /invalid_symlink
/invalid_symlink: broken symbolic link to /mnt/sda1/invalid_folder
guest@rava:~$ echo -n "/mnt/sda1/invalid_folder" |wc
      0       1      24
guest@rava:~$ 

The length of a symlink is always the length of the path it contains, in this example case: "/mnt/sda1/invalid_folder" is 24 characters as echo -n "/mnt/sda1/invalid_folder" |wc tells you, and that is what stat also reports as its size. But that is only the entry in the directory, that space in the directory - especially when it's the root directory of any partition - exists already, but is unused.
Using that directory entry makes your partition not any fuller than it was previously.

and the Blocks: is still zero, since symlinks also take no extra space in your file system. Similar with empty folders or empty files. On a real filesystem empty folders take at least one block of space (fo their currently unused directory entries), but in the aufs that is different and as you can see they would take 40 bytes - if they contained any files or folders, but as they do not - their Blocks: is also still zero.

[Ed_P]

Code: Select all

guest@rava:~$ echo -n "/mnt/sda1/invalid_folder" |wc

An interesting piece of code rava.

Code: Select all

guest@porteus:~$ echo -n /mnt/loop/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/moz-extension+++a0f08673-3fa9-4163-8fcb-32ed4b44fa2d^userContextId=4294967295/idb/3647222921wleabcEoxlt-eengsairo.files/.wh.604 |wc
      0       1     206
guest@porteus:~$ 

So 206 * 900 = 185,400 -> 185KB

[Rava]

An interesting piece of code rava.

At times the most easy and trivial one-liner can be very helpful.
Usually I prefer doing repetitive tasks via shell than via a GUI program, in case someone not yet realized that. And also using scripts.

Code: Select all

guest@porteus:~$ echo -n /mnt/loop/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/moz-extension+++a0f08673-3fa9-4163-8fcb-32ed4b44fa2d^userContextId=4294967295/idb/3647222921wleabcEoxlt-eengsairo.files/.wh.604 |wc
      0       1     206
guest@porteus:~$ 

So 206 * 900 = 185,400 -> 185KB

What does

Code: Select all

stat /mnt/loop/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/moz-extension+++a0f08673-3fa9-4163-8fcb-32ed4b44fa2d^userContextId=4294967295/idb/3647222921wleabcEoxlt-eengsairo.files/.wh.604

report as size and as blocks?

The "900" is the number of such files in your /mnt/loop/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/moz-extension* folder I presume?

[Ed_P]

The "900" is the number of such files in your /mnt/loop/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/moz-extension* folder I presume?

No, it is an estimate of the number of .wh. files I usually find.

Code: Select all

guest@porteus:~$ stat /mnt/loop/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/moz-extension+++a0f08673-3fa9-4163-8fcb-32ed4b44fa2d^userContextId=4294967295/idb/3647222921wleabcEoxlt-eengsairo.files/.wh.604
stat: cannot statx '/mnt/loop/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/moz-extension+++a0f08673-3fa9-4163-8fcb-32ed4b44fa2d^userContextId=4294967295/idb/3647222921wleabcEoxlt-eengsairo.files/.wh.604': No such file or directory
guest@porteus:~$ 
guest@porteus:~$ su
Password: 
root@porteus:/home/guest# stat /mnt/loop/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/moz-extension+++a0f08673-3fa9-4163-8fcb-32ed4b44fa2d^userContextId=4294967295/idb/3647222921wleabcEoxlt-eengsairo.files/.wh.604
stat: cannot statx '/mnt/loop/home/guest/.mozilla/firefox/ple67xm3.default-release/storage/default/moz-extension+++a0f08673-3fa9-4163-8fcb-32ed4b44fa2d^userContextId=4294967295/idb/3647222921wleabcEoxlt-eengsairo.files/.wh.604': No such file or directory
root@porteus:/home/guest# 

Apparently you can not stat a deleted file. And it still shows in my find /mnt/loop -name .wh.* list.