[HOWTO] make a xzm module driver for NVIDIA video card

Post#31 by **Blaze** » 24 Feb 2019, 17:41

I updated my script again.

fulalas, I don't know why your $PORTDIR variable is empty. Any suggestion are welcome.

Has anyone tested my updated script?

Post#32 by **Rava** » 23 Mar 2019, 08:04

Blaze, Hi, I suggest using a more or less uniq name for temp folders or temp files, using the same /tmp/nvidia/ folder can get the user into trouble when he has started a script already but that is halted and he starts one anew. (One could argue that this would then be the users fault, but a script better be coded to handle some user errors than not handle any possible user errors.)

Better than using

Code: Select all

TEMPFOLDER=/tmp/nvidia

is this:

Code: Select all

TEMPFOLDER=/tmp/nvidia$$

Example:

guest@porteus:/$ echo /tmp/usm
/tmp/usm
guest@porteus:/$ echo /tmp/usm$$
/tmp/usm3000

3000 is the current shell PID or Process ID or some claim it is called "Process Identifier".
And since only one process can have a certain PID, 3000 in this case, only that process would create and use the folder /tmp/usm3000 and all other scripts started later or prior would use a different folder name.

_______________________________

Also, this fails with my setup:

Code: Select all

# Check blacklist.xzm in $PORTDIR/base
if [[ -f "$PORTDIR/base/blacklist.xzm" ]]; then
    mkdir /tmp/nvidia
    xzm2dir "$PORTDIR/base/blacklist.xzm" /tmp/nvidia &>/dev/null
else
    echo "$PORTDIR/base/blacklist.xzm" does not exist
    exit 1
fi

Reason? I don't have blacklist in $PORTDIR/base/blacklist.xzm.
I use symlinks to the needed files, these sit in /optional/ and all start with XXX, like so:

Code: Select all

XXX05-devel_V20190224.xzm
XXX06-crippled_sources-4.20.12-64bit.xzm
XXXblacklist.xzm

so that I can use this cheat code for running the Nvidia driver compile session without having to copy or move any files around on my boot partition:

Code: Select all

noload=xfce load=XXX 3

Post#33 by **Ed_P** » 23 Mar 2019, 15:29

Instead of filling the /tmp folder with junk I recommend the script use this, if it doesn't already.

Code: Select all

if [ -d /tmp/nvidia/ ]; then
   rm -rf /tmp/nvidia
fi
mkdir -p /tmp/nvidia

Post#34 by **Blaze** » 23 Mar 2019, 16:06

Hi Rava, thanks for your suggestions.
I did a couple of changes in my script and I doubt a little how the first structure (# Check blacklist.xzm in $PORTDIR/{base,optional}) will behave.

Code: Select all

#!/bin/bash
# Build of NVIDIA video driver, cleaning and compressing to xzm module for Porteus by Blaze
# You can use this script for both architectures i586 or x86_64
# Version 2019-03-29

TMP=${TMP:-/tmp/nvidia-$$}

# Root check
if [ `whoami` != "root" ]; then
    echo -e "\n\e[1m\e[31mOnly root can run this.\e[0m\n"
    exit 1
fi

# Check blacklist.xzm in $PORTDIR/{base,optional}
if [[ -f "$PORTDIR/base/blacklist.xzm" ]]; then
    mkdir $TMP
    xzm2dir "$PORTDIR/base/blacklist.xzm" $TMP &>/dev/null
elif [[ -f "$PORTDIR/optional/*blacklist.xzm" ]]; then
    mkdir $TMP
    xzm2dir "$PORTDIR/optional/*blacklist.xzm" $TMP &>/dev/null
else
    ls "$PORTDIR/base/blacklist.xzm" > /dev/null
    ls "$PORTDIR/optional/*blacklist.xzm" > /dev/null
    echo -e "\n\e[1m\e[31m"'blacklist.xzm not found in Porteus /base or /optional. Abort!'"\e[0m\n"
    unset TMP
    exit 1
fi

# Сleaning and compressing to xzm module
echo -e '\n\033[1mCompression of memory changes to the module /tmp/nvidia.xzm\033[0m\n'
sync; echo 3 > /proc/sys/vm/drop_caches
mksquashfs /mnt/live/memory/changes /tmp/nvidia.xzm -comp xz -b 128K -Xbcj x86 -noappend 2>/dev/null
cd /tmp
echo -e '\n\033[1mExtracting changes to /tmp/nvidia\033[0m\n'
xzm2dir nvidia.xzm $TMP 2>/dev/null
rm -rf $TMP/{.cache,dev,home,mnt,root,run,tmp,var}
find $TMP -type f -maxdepth 1 -delete
find $TMP -type l -maxdepth 1 -delete
find $TMP/etc/ -type f -maxdepth 1 -delete
find $TMP/etc/ -type d ! -iname 'modprobe.d' ! -iname 'OpenCL' ! -iname 'vulkan' ! -iname 'X11' ! -iname 'etc' -maxdepth 1 -exec rm -rf '{}' '+'
rm -f $TMP/etc/X11/xorg.conf.nvidia-xconfig-original
rm -rf $TMP/lib/firmware
rm -f $TMP/lib/modules/*porteus/modules.*
rm -rf $TMP/usr/{man,src}
rm -f $TMP/usr/bin/gnome-keyring-daemon
rm -rf $TMP/usr/lib/{gio,gtk-2.0,gtk-3.0}
rm -f $TMP/usr/lib/{libXvMCgallium.so.1,libbrscandec2.so.1,libgsm.so.1,libudev.so.1,libunrar.so.5}
rm -rf $TMP/usr/lib64/{gio,gtk-2.0,gtk-3.0}
rm -f $TMP/usr/lib64/{libXvMCgallium.so.1,libbrscandec2.so.1,libgsm.so.1,libudev.so.1,libunrar.so.5}
rm -rf $TMP/usr/share/{glib-2.0,mime,pixmaps}
rm -f $TMP/usr/{,local/}share/applications/mimeinfo.cache
rm -rf $TMP/usr/share/doc/NVIDIA_GLX-1.0/{html,sample,LICENSE,NVIDIA_Changelog,README.txt}
dir2xzm $TMP /tmp/08-nvidia-$(ls /tmp/*.run | basename -s .run *.run | cut -d '-' -f4)-k.$(uname -r)-$(cat /etc/porteus-version | cut -d '-' -f2)-$(arch).xzm 2>/dev/null
mv /tmp/08-nvidia-$(ls /tmp/*.run | basename -s .run *.run | cut -d '-' -f4)-k.$(uname -r)-$(cat /etc/porteus-version | cut -d '-' -f2)-$(arch).xzm $MODDIR

# Removing $PORTDIR/base/blacklist.xzm
if [[ -f "$PORTDIR/base/blacklist.xzm" ]]; then
    echo -e "\n\033[1mRemoving $PORTDIR/base/blacklist.xzm\033[0m\n"
    rm -f $PORTDIR/base/blacklist.xzm
fi

echo -e "\n\t\e[1m\e[32m[DONE]\e[0m \e[1mYou can find your nvidia xzm module in \e[96m$MODDIR\e[0m"
echo -e "\e[1m\t\t>>> Now you can reboot Porteus via \e[92mreboot\e[0m \e[1mcommand <<<\e[0m\n"

unset TMP

Can you test it and report about success or fail?

Thanks.

Post#35 by **Rava** » 23 Mar 2019, 18:56

I will test it with a test version that lacks the 340.107 driver (kernel 4.20.12) soon and give you the heads up.

Blaze wrote: ↑
23 Mar 2019, 16:06
dir2xzm /mnt/live/memory/changes /tmp/nvidia.xzm 2>/dev/null
cd /tmp
echo -e '\n\033[1mExtracting changes to /tmp/nvidia\033[0m\n'
xzm2dir nvidia.xzm $TMP 2>/dev/null

The issue I have with this attempt is that it could break the system:
In my current system doing the above would fill up the / filesystem and that gets the whole system in an unstable state.
Especially folks with not so much RAM are at an disadvantage.

My suggestion: introducing of a parameter that lets the nvidia.sh script move the temp folder from /tmp/ to a place inside any Linux partition and folder the user chooses to.
In my book, I would even use a quick and dirty hack checking if its a Linux partition and not FAT, VFAT or NTFS or any other filesystem that not supports read only files and folders like a Linux partition does: create a file in it with certain properties that only would work on a Linux partition and then check if the properties of the created empty dummy file are as they are told to be.
If they are: presume the given folder for temporary files and subfolders is indeed on a Linux partition.
You just have to tweak the script into either using /tmp/nvidia-$$ or "$1", if $1 is a folder on a Linux partition, see above.

Post#36 by **Rava** » 25 Mar 2019, 01:42

I also suggest adding version info, and be it only like so:

Code: Select all

#Version 2019-03-25

________________________

Also, I looked into your above code.

Code: Select all

    ls "$PORTDIR/base/blacklist.xzm" > /dev/null
    ls "$PORTDIR/optional/*blacklist.xzm" > /dev/null
    unset TMP
    exit 1

This will exit without giving the user any info why it was exited, would look like a crash for the user.

And what is the purpose of the two ls that only got redirected to /dev/null?

Better would be e.g. this:

Code: Select all

    echo -e "\n\e[1m\e[31m"'*blacklist.xzm not found in Porteus /base or /optional. Abort!'"\e[0m\n"
    unset TMP
    exit 1

Post#37 by **Rava** » 25 Mar 2019, 15:54

Update

I could not compile the driver:

Code: Select all

         NVIDIA Accelerated Graphics Driver for Linux-x86_64 (340.107)

  The CC version check failed:

  The compiler used to compile the kernel (gcc 8.2) does not exactly match the
  current compiler (gcc 8.3).  The Linux 2.6 kernel module loader rejects
  kernel modules built with a version of gcc that does not exactly match that
  of the compiler used to build the running kernel.

  If you know what you are doing you can either ignore the CC version check
  and continue installation, or abort installation, set the CC environment
  variable to the name of the compiler used to compile your kernel, and
  restart installation.

                Ignore CC version check     Abort installation

  NVIDIA Software Installer for Unix/Linux                      www.nvidia.com

It should have been the driver for 4.20.12...

Post#38 by **Blaze** » 25 Mar 2019, 17:41

Rava, about gcc 8.2

1) you are must to use this kernel 4.20.10 by neko with with 05-devel.xzm and 06-crippled_sources-4.20.10-64bit.xzm (I think you are know were they can be found), because this kernel builded under gcc 8.2.
2) or you can try latest kernel 5.x.x (gcc 8.3) by neko with latest 05-devel.xzm (include gcc 8.3) and 06-crippled_sources. If this is of course compatible with your video card.

Rava wrote: ↑
25 Mar 2019, 01:42
And what is the purpose of the two ls that only got redirected to /dev/null?

if file exists you'll don't get error, but if the file does not exist you'll get error|message about this.

For example. You are have XXXblacklist.xzm in $PORTDIR/optional and not have blacklist.xzm in $PORTDIR/base

Code: Select all

# ls "$PORTDIR/base/blacklist.xzm" > /dev/null
/bin/ls: unable to access '/mnt/sdb1/porteus/base/blacklist.xzm': No such file or directory

# ls "$PORTDIR/optional/*blacklist.xzm" > /dev/null

You don't get error, because XXXblacklist.xzm placed in $PORTDIR/optional

Post#39 by **Rava** » 25 Mar 2019, 17:46

Blaze wrote: ↑
25 Mar 2019, 17:41
if file exists you'll don't get error, but if the file does not exist you'll get error|message about this.

For example. You are have XXXblacklist.xzm in $PORTDIR/optional and not have blacklist.xzm in $PORTDIR/base
Code: Select all
# ls "$PORTDIR/base/blacklist.xzm" > /dev/null
/bin/ls: unable to access '/mnt/sdb1/porteus/base/blacklist.xzm': No such file or directory

# ls "$PORTDIR/optional/*blacklist.xzm" > /dev/null

You don't get error, because XXXblacklist.xzm placed in $PORTDIR/optional

Silly me. You are right, still I suggest adding an additional error message.

Post#40 by **Blaze** » 25 Mar 2019, 18:47

Ok, I updated post #34

Thanks.

Post#41 by **Ed_P** » 25 Mar 2019, 18:55

What is someone renames their blacklist.xzm file blacklist-x84_64.xzm? Or blacklist-19-03-25.xzm? Or BlackList.xzm? Or adds their email address blacklist as blacklist.xzm to their modules folder?

Maybe just testing for blacklist.xzm is fine and the error msg if not found should be changed.

"Standard Porteus blacklist.xzm module not found."

Post#42 by **Rava** » 25 Mar 2019, 22:39

Ed_P wrote: ↑
25 Mar 2019, 18:55
Or adds their email address blacklist as blacklist.xzm to their modules folder?

Then the user definitely has not read the instructions (available to my knowledge in Russian and English), and what happens when he starts a script but has not read the long detailed instructions that are an integral part of the script - then anything "weird" or "bad" that happens is solely the users fault for not RTFM.

Just my 2 cents. The script is okay when it comes to base/blacklist.xzm.

Post#43 by **Ed_P** » 25 Mar 2019, 23:38

Oh, I don't know Rava.

I heard someone say it wouldn't support their renamed XXXblacklist.xzm file. I think they read the manual.

Post#44 by **Rava** » 27 Mar 2019, 06:30

^

The reply to this and later replies have been moved to a more appropriate thread here: Does irony translate well into email or forum posts?

Post#45 by **Blaze** » 30 Mar 2019, 18:07

I some optimized my script. Please test it on your real iron

Code: Select all

#!/bin/bash
# Build of NVIDIA video driver, cleaning and compressing to xzm module for Porteus by Blaze
# You can use this script for both architectures i586 or x86_64
# Version 2019-04-20

TMP=${TMP:-/tmp/nvidia-$$}
NV=${NV:-08-nvidia-$(ls /tmp/*.run | basename -s .run *.run | cut -d '-' -f4  | sed -n 1p)-k.$(uname -r)-$(cat /etc/porteus-version | cut -d '-' -f2)-$(arch).xzm}

# Root check
if [ `whoami` != "root" ]; then
    echo -e "\n\e[1m\e[31mOnly root can run this.\e[0m\n"
    exit 1
fi

# Check blacklist.xzm in $PORTDIR/{base,optional}
if [[ -f "$PORTDIR/base/blacklist.xzm" ]]; then
    mkdir $TMP
    xzm2dir "$PORTDIR/base/blacklist.xzm" $TMP &>/dev/null
elif [[ -f "$PORTDIR/optional/*blacklist.xzm" ]]; then
    mkdir $TMP
    xzm2dir "$PORTDIR/optional/*blacklist.xzm" $TMP &>/dev/null
else
    ls "$PORTDIR/base/blacklist.xzm" > /dev/null
    ls "$PORTDIR/optional/*blacklist.xzm" > /dev/null
    echo -e "\n\e[1m\e[31m"'blacklist.xzm not found in Porteus /base or /optional. Abort!'"\e[0m\n"
    unset TMP
    exit 1
fi

# Selective compression and extraction of memory changes for NVIDIA
cd /tmp
sync; echo 3 > /proc/sys/vm/drop_caches
echo -e '\n\033[1m\e[32m[*]\e[0m\033[1m Compression of memory changes for NVIDIA to /tmp/nvidia.tar.gz\033[0m'
tar czf nvidia.tar.gz --exclude={"*/.*","*/.wh.*",".cache","dev","home","mnt","opt","root","run","tmp","var","etc/bootcmd.cfg","etc/ld.so.cache","etc/fstab","etc/random-seed","etc/cups","etc/udev","etc/profile.d","etc/porteus","etc/X11/xorg.conf.nvidia-xconfig-original","lib/firmware","lib/modules/*porteus/modules.*","usr/man","usr/src","usr/bin/gnome-keyring-daemon","usr/lib/gio","usr/lib/gtk-2.0","usr/lib/gtk-3.0","usr/lib/libXvMCgallium.so.1","usr/lib/libbrscandec2.so.1","usr/lib/libgsm.so.1","usr/lib/libudev.so.1","usr/lib/libunrar.so.5","usr/lib64/gio","usr/lib64/gtk-2.0","usr/lib64/gtk-3.0","usr/lib64/libXvMCgallium.so.1","usr/lib64/libbrscandec2.so.1","usr/lib64/libgsm.so.1","usr/lib64/libudev.so.1","usr/lib64/libunrar.so.5","usr/local","usr/share/glib-2.0","usr/share/mime","usr/share/pixmaps","usr/share/applications/mimeinfo.cache","usr/local/share/applications/mimeinfo.cache","usr/share/doc/NVIDIA_GLX-1.0/html","usr/share/doc/NVIDIA_GLX-1.0/sample","usr/share/doc/NVIDIA_GLX-1.0/LICENSE","usr/share/doc/NVIDIA_GLX-1.0/NVIDIA_Changelog","usr/share/doc/NVIDIA_GLX-1.0/README.txt"} -C /mnt/live/memory changes
echo -e "\n\033[1m\e[32m[*]\e[0m\033[1m Extraction of memory changes for NVIDIA from /tmp/nvidia.tar.gz to $TMP\033[0m\n"
tar xf nvidia.tar.gz --strip 1 -C $TMP
rm -f /tmp/nvidia.tar.gz

# Сleaning and compressing to xzm module
echo -e "\n\033[1m\e[32m[*]\e[0m\033[1m Сleaning $TMP directory from junk files\033[0m\n"
rm -rf $TMP/{.cache,dev,home,mnt,opt,root,run,tmp,var}
find $TMP -type f -maxdepth 1 -delete
find $TMP -type l -maxdepth 1 -delete
find $TMP/etc/ -type f -maxdepth 1 -delete
find $TMP/etc/ -type d ! -iname 'modprobe.d' ! -iname 'OpenCL' ! -iname 'vulkan' ! -iname 'X11' ! -iname 'etc' -maxdepth 1 -exec rm -rf '{}' '+'
rm -f $TMP/etc/X11/xorg.conf.nvidia-xconfig-original
rm -rf $TMP/lib/firmware
rm -f $TMP/lib/modules/*porteus/modules.*
rm -rf $TMP/usr/{man,src}
rm -f $TMP/usr/bin/gnome-keyring-daemon
rm -rf $TMP/usr/lib/{gio,gtk-2.0,gtk-3.0}
rm -f $TMP/usr/lib/{libXvMCgallium.so.1,libbrscandec2.so.1,libgsm.so.1,libudev.so.1,libunrar.so.5}
rm -rf $TMP/usr/lib64/{gio,gtk-2.0,gtk-3.0}
rm -f $TMP/usr/lib64/{libXvMCgallium.so.1,libbrscandec2.so.1,libgsm.so.1,libudev.so.1,libunrar.so.5}
rm -rf $TMP/usr/local
rm -rf $TMP/usr/share/{glib-2.0,mime,pixmaps}
rm -f $TMP/usr/{,local/}share/applications/mimeinfo.cache
rm -rf $TMP/usr/share/doc/NVIDIA_GLX-1.0/{html,sample,LICENSE,NVIDIA_Changelog,README.txt}
echo -e "\n\033[1m\e[32m[*]\e[0m\033[1m Compressing $TMP to $MODDIR/$NV module\033[0m\n"
dir2xzm $TMP $MODDIR/$NV 2>/dev/null
rm -rf $TMP

# Removing $PORTDIR/base/blacklist.xzm
if [[ -f "$PORTDIR/base/blacklist.xzm" ]]; then
    echo -e "\n\033[1m\e[32m[*]\e[0m\033[1m Removing $PORTDIR/base/blacklist.xzm\033[0m\n"
    rm -f $PORTDIR/base/blacklist.xzm
fi

echo -e "\n\e[1m\e[32m[DONE]\e[0m \e[1mYou can find your nvidia xzm module in \e[96m$MODDIR\e[0m"
echo -e "\e[1m>>> Now you can reboot Porteus via \e[92mreboot\e[0m \e[1mcommand <<<\e[0m\n"

unset TMP
unset NV