NTFS3 Kernel Module

[Blaze]

build the ntfs3 on 5.4.57-porteus can you use the make -d debug seem that file kernel are too old for the ntfs3 build

Seems you are right, because I have faile with my script and kernel 5.4.65

Code: Select all

#!/bin/bash
# ntfs3-dkms driver
# NTFS read-write driver GPL implementation by Paragon Software.
# Current version works with NTFS (including v3.1), normal/compressed/sparse files and supports journal replaying.
# Slackware build script for broadcom-wl-dkms
# base on https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=ntfs3-dkms
# ported for Slackware by Blaze
# depends=(dkms)

PRGNAM=ntfs3
VERSION=${VERSION:-v6.0.0}
KERNEL=${KERNEL:-`uname -r`}
BUILD=${BUILD:-1}
TAG=${TAG:-bl}

if [ -z "$ARCH" ]; then
  case "$( uname -m )" in
    i?86) ARCH=i586 ;;
    arm*) ARCH=arm ;;
       *) ARCH=$( uname -m ) ;;
  esac
fi

CWD=$(pwd)
TMP=${TMP:-/tmp/SBo}
PKG=$TMP/package-$PRGNAM
OUTPUT=${OUTPUT:-/tmp}

if [ "$ARCH" = "i586" ]; then
  SLKCFLAGS="-O2 -march=i586 -mtune=i686"
  LIBDIRSUFFIX=""
elif [ "$ARCH" = "i686" ]; then
  SLKCFLAGS="-O2 -march=i686 -mtune=i686"
  LIBDIRSUFFIX=""
elif [ "$ARCH" = "x86_64" ]; then
  SLKCFLAGS="-O2 -fPIC"
  LIBDIRSUFFIX="64"
else
  SLKCFLAGS="-O2"
  LIBDIRSUFFIX=""
fi

set -e

rm -rf $PKG $TMP/$PRGNAM-$VERSION
mkdir -p $TMP $PKG/usr/src/ntfs3-${VERSION} $OUTPUT /usr/src/ntfs3-${VERSION} "${TMP}/${PRGNAM}-${VERSION}"

cd "${TMP}/${PRGNAM}-${VERSION}"

lynx --source https://aur.archlinux.org/cgit/aur.git/plain/dkms.conf?h=ntfs3-dkms > ${TMP}/${PRGNAM}-${VERSION}/dkms.conf
lynx --source https://aur.archlinux.org/cgit/aur.git/plain/Makefile.patch?h=ntfs3-dkms > ${TMP}/${PRGNAM}-${VERSION}/Makefile.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308074/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~1.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308079/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~2.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308077/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~3.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308073/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~4.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308069/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~5.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308070/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~6.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308078/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~7.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308067/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~8.patch

sed -i 's|$(KVERSION)|KERNEL|g' "${TMP}/${PRGNAM}-${VERSION}/Makefile.patch"
sed -i "s|KERNEL|${KERNEL}|g" "${TMP}/${PRGNAM}-${VERSION}/Makefile.patch"

for patch in "$VERSION~"*
do
    patch -p3 -N -i "$patch"
done
    patch -p1 -N -i "Makefile.patch"

cp -r "${TMP}/${PRGNAM}-${VERSION}" "/usr/src/"
install -Dm644 "${TMP}/${PRGNAM}-${VERSION}/dkms.conf" "/usr/src/ntfs3-${VERSION}/dkms.conf"

cd /usr/src/ntfs3-v6.0.0/
dkms install --no-depmod -m ntfs3 -v $VERSION -k `uname -r`
depmod `uname -r`

#cd "$OUTPUT/$PRGNAM-$VERSION-$KERNEL-$ARCH-$BUILD$TAG"
#/sbin/makepkg -l y -c n $OUTPUT/$PRGNAM-$VERSION-$KERNEL-$ARCH-$BUILD$TAG.${PKGTYPE:-txz}

Code: Select all

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...
KVERSION=5.4.65-porteus CONFIG_NTFS3_FS=m make KDIR=/lib/modules/5.4.65-porteus/build...(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.4.65-porteus (x86_64)
Consult /var/lib/dkms/ntfs3/v6.0.0/build/make.log for more information.

Code: Select all

# cat /var/lib/dkms/ntfs3/v6.0.0/build/make.log
DKMS make.log for ntfs3-v6.0.0 for kernel 5.4.65-porteus (x86_64)
Пн сен 21 18:17:23 MSK 2020
make -C /lib/modules/5.4.65-porteus/build M=/var/lib/dkms/ntfs3/v6.0.0/build modules
make[1]: вход в каталог «/usr/src/linux-5.4.65»
  CC [M]  /var/lib/dkms/ntfs3/v6.0.0/build/bitfunc.o
  CC [M]  /var/lib/dkms/ntfs3/v6.0.0/build/bitmap.o
/var/lib/dkms/ntfs3/v6.0.0/build/bitmap.c: В функции «wnd_rescan»:
/var/lib/dkms/ntfs3/v6.0.0/build/bitmap.c:558:4: ошибка: неявная декларация функции «page_cache_readahead_unbounded» [-Werror=implicit-function-declaration]
  558 |    page_cache_readahead_unbounded(mapping, NULL,
      |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: некоторые предупреждения считаются ошибками
make[2]: *** [scripts/Makefile.build:266: /var/lib/dkms/ntfs3/v6.0.0/build/bitmap.o] Ошибка 1
make[1]: *** [Makefile:1729: /var/lib/dkms/ntfs3/v6.0.0/build] Ошибка 2
make[1]: выход из каталога «/usr/src/linux-5.4.65»
make: *** [Makefile:14: all] Ошибка 2

The same error NTFS3 Kernel Module (Пост beny #78750)

But, you can use (tweak) my script for build ntfs3 module on a new kernel

[beny]

hi, seem we have to wait for aufs patch for the 5.8x kernel to build for porteus,with arch or current no problem with dkms or not but lts kernel is out of the build

[Blaze]

beny,

Code: Select all

+ dkms install --no-depmod -m ntfs3 -v v6.0.0 -k 5.7.19-porteus

Creating symlink /var/lib/dkms/ntfs3/v6.0.0/source ->
                 /usr/src/ntfs3-v6.0.0

DKMS: add completed.

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...
KVERSION=5.7.19-porteus CONFIG_NTFS3_FS=m make KDIR=/lib/modules/5.7.19-porteus/build...(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.7.19-porteus (x86_64)
Consult /var/lib/dkms/ntfs3/v6.0.0/build/make.log for more information.

Code: Select all

# cat /var/lib/dkms/ntfs3/v6.0.0/build/make.log
DKMS make.log for ntfs3-v6.0.0 for kernel 5.7.19-porteus (x86_64)
Вт сен 22 13:39:47 MSK 2020
make -C /lib/modules/5.7.19-porteus/build M=/var/lib/dkms/ntfs3/v6.0.0/build modules
make[1]: вход в каталог «/usr/src/linux-5.7.19»
make[2]: Предупреждение: время изменения файла «include/config/auto.conf» находится в будущем (7945)
  CC [M]  /var/lib/dkms/ntfs3/v6.0.0/build/bitfunc.o
  CC [M]  /var/lib/dkms/ntfs3/v6.0.0/build/bitmap.o
/var/lib/dkms/ntfs3/v6.0.0/build/bitmap.c: В функции «wnd_rescan»:
/var/lib/dkms/ntfs3/v6.0.0/build/bitmap.c:558:4: ошибка: неявная декларация функции «page_cache_readahead_unbounded» [-Werror=implicit-function-declaration]
  558 |    page_cache_readahead_unbounded(mapping, NULL,
      |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: некоторые предупреждения считаются ошибками
make[2]: *** [scripts/Makefile.build:267: /var/lib/dkms/ntfs3/v6.0.0/build/bitmap.o] Ошибка 1
make[1]: *** [Makefile:1732: /var/lib/dkms/ntfs3/v6.0.0/build] Ошибка 2
make[1]: выход из каталога «/usr/src/linux-5.7.19»
make: *** [Makefile:14: all] Ошибка 2

[babam]

Here is NTFS3 for Porteus 5.0 RC2 (Kernel 5.4.57) https://github.com/LGA1150/ntfs3-oot

Code: Select all

# mount -t ntfs3 /dev/sdXX /mnt/sdXX -o noatime,nodiratime,suid,dev,exec,async,uid=1000,gid=100,umask=022,fmask=022,dmask=022,discard,nls=utf8,prealloc

[babam]

Pros:
1. Very fast, much faster than ntfs-3g.
2. Prevents fragmentation when writing files, whereas ntfs-3g generates a lot of fragmentation.
3. Low in CPU and RAM usage.

Cons:
1. NTFS3 has not been tested for stability, reliability and security. NTFS-3G has been tested for stability and safety.

[rych]

Does Porteus 5.0 by default use the new ntfs3 kernel driver or is it still FUSE ntfs-3g?

Starting with the Linux 5.15 kernel, NTFS is finally being fully supported in Linux.

New NTFS Driver in Kernel 5.15

https://www.kernel.org/doc/html/latest/ ... ntfs3.html

Slackware Forum: Kernel 5.15 : ntfs3 vs ntfs-3g

[ncmprhnsbl]

Does Porteus 5.0 by default use the new ntfs3 kernel driver or is it still FUSE ntfs-3g?

still FUSE ntfs-3g

[burdi01]

Does Porteus 5.0 by default use the new ntfs3 kernel driver or is it still FUSE ntfs-3g?

still FUSE ntfs-3g

but you can use the ntfs3 driver: see https://www.linuxquestions.org/question ... 175702945/

[rych]

still FUSE ntfs-3g

Maybe that's why it's slow. I find that the write speed off the host system's NTFS on NVMe SSD is twice slower than on the ext4 on an NVMe SSD USB 3 10Gbit thumbdrive.

Since the ntfs3 kernel driver was introduced in 5.15, there have been bugfixes and feature updates: for example in 5.19, in 6.2, in 6.4 and in 6.6 https://lore.kernel.org/ntfs3/

So, perhaps the early reports of (rare but shocking) NTFS file system corruption have now been addressed and the bugs fixed? I'll be trying to remount some of the less important, always backed-up NTFS partitions with ntfs3 driver instead of ntfs-3g and will test. Two questions: how do I it properly? Is it just

Code: Select all

umount /dev/sda4
mount -tntfs3 /dev/sda4 /mnt

Second question/request to ncmprhnsbl: thanks for updating us to Porteus 5.01 with the kernel 6.5.5, could you offer kernel updates as Updates to Porteus like you do with modules? I know it involves rebuilding Nvidia driver etc. but perhaps when it hits a 6.6 and other milestones? It's because the ntfs3 bugfixing is still in progress and some are checked in for 6.6 and so on.

[rych]

you can use the ntfs3 driver

How? Its module doesn't seem to be loaded by default and can't be loaded:

Code: Select all

root@porteus:~# lsmod | grep ntfs
root@porteus:~# modprobe ntfs3
root@porteus:~# lsmod | grep ntfs
root@porteus:~# 
root@porteus:/mnt/sdb3# systool -v -m ntfs3
Error opening module ntfs3

Perhaps it's somehow active anyway:

Code: Select all

root@porteus:/mnt/sdb3# modinfo ntfs3
name:           ntfs3
filename:       (builtin)
alias:          fs-ntfs3
author:         Konstantin Komarov
compression:    Read-only lzx/xpress compression included
behaviour:      Enabled Linux POSIX ACLs support
description:    ntfs3 read/write filesystem
license:        GPL
file:           fs/ntfs3/ntfs3

There are ntfs3 messages in:

Code: Select all

root@porteus:~# dmesg | grep ntfs3
[    0.286414] ntfs3: Max link count 4000
[    0.286415] ntfs3: Enabled Linux POSIX ACLs support
[    0.286416] ntfs3: Read-only LZX/Xpress compression included
[ 2410.517027] ntfs3: sdb3: It is recommened to use chkdsk.
[ 2412.925950] ntfs3: sdb3: volume is dirty and "force" flag is not set!

It's a known file system according to

Code: Select all

root@porteus:/mnt/sdb3# cat /proc/filesystems | grep ntfs
	ntfs
	ntfs3

If anyone manages to mount an NTFS with ntfs3, could you teach me. For now I'm getting these results:

Code: Select all

root@porteus:/mnt/sdb3# mount -fv -tntfs3 /dev/sdb3
mount: /dev/sdb3 mounted on /mnt/sdb3.
root@porteus:/mnt/sdb3# umount -v /dev/sdb3
umount: /dev/sdb3: not mounted.
root@porteus:/mnt/sdb3# mount -v -tntfs3 /dev/sdb3
mount: /mnt/sdb3: wrong fs type, bad option, bad superblock on /dev/sdb3, missing codepage or helper program, or other error.

I feel like there are rumors of ntfs3 on my system but the module itself is not actually included with the kernel (Porteus 5.01)?!

[rych]

OK, after

Code: Select all

ntfsfix --clear-dirty /dev/sdb3

this works:

Code: Select all

mount -t ntfs3 /dev/sdb3
findmnt | grep ntfs
└─/mnt/sdb3  /dev/sdb3 ntfs3 rw,nosuid,nodev,noatime,nodiratime,uid=0,gid=0,iocharset=utf8

[fulalas]

OK, after

Code: Select all

ntfsfix --clear-dirty /dev/sdb3

this works:

Nice! Now, the big question: what about performance?

[rych]

Now, the big question: what about performance?

Looks good already: Booting from NVMe USB enclosure (Post by rych #96861). Will try on larger faster disks soon -- a bit afraid, so will backup first

[rych]

ncmprhnsbl, Now that I've tested it for a while with no stability/integrity issues whatsoever, could you help me replace ntfs-3g with ntfs3 in mounting all NTFS partitions at boot please? I think I need to modify these 2 lines in the finit:

Code: Select all

ntfs-3g /dev/$1 /mnt/$1 -o $MOPT,big_writes 2>/dev/null || rmdir /mnt/$1
...
ntfs-3g /dev/$x /mnt/$x -o $MOPT,big_writes 2>/dev/null || { sed -i "/$x /d" /etc/fstab; rmdir /mnt/$x; }

Firstly: why there are two occurrences of that mount command and which one applies?

Secondly, how do I properly rewrite it using ntfs3?

At the moment, I re-mount a posteriori some partitions that I still can, that are not "busy", inheriting the same mount options and only changing the driver, with e.g.,

Code: Select all

umount /dev/sdb3
mount -t ntfs3 /dev/sdb3

... and it works: options and the mount point are read from fstab I guess. I could add some ntfs3 specific options too...

But I'd really like for all my NTFS partitions to be mounted from the start with ntfs3 if possible.

[rych]

I suggest replacing

Code: Select all

ntfs-3g /dev/$1 /mnt/$1 -o $MOPT,big_writes 2>/dev/null || rmdir /mnt/$1

with

Code: Select all

mount -n -t ntfs3 /dev/$1 /mnt/$1 -o $MOPT 2>/dev/null || rmdir /mnt/$1

Is that okay? Strange that that errors are not diplayed. There could be an important error in case of the ntfs3 mounting: for example failing due to the dirty bit. Instead the commands above remove /mnt/$1 in case of error -- is that necessary?

Anyway, treading carefully, can I just replace everywhere
ntfs-3g /dev/$1 /mnt/$1 -o $MOPT,big_writes
-->
mount -n -t ntfs3 /dev/$1 /mnt/$1 -o $MOPT

(Also an interesting option should be -n is used, why?)


Another relevant place is in the linuxrc file, and I suggest this change:

Code: Select all

sed -i 's/ ntfs / ntfs-3g /g' /etc/fstab

-->

Code: Select all

sed -i 's/ ntfs / ntfs3 /g' /etc/fstab

(this command searches-and-replaces in the /etc/fstab)