NTFS3 Kernel Module

Technical issues/questions of an intermediate or advanced nature.
User avatar
Blaze
DEV Team
DEV Team
Posts: 3869
Joined: 28 Dec 2010, 11:31
Distribution: ⟰ Porteus current ☯ all DEs ☯
Location: ☭ Russian Federation, Lipetsk region, Dankov
Contact:

NTFS3 Kernel Module

Post#16 by Blaze » 21 Sep 2020, 15:24

beny wrote:
17 Sep 2020, 18:38
build the ntfs3 on 5.4.57-porteus can you use the make -d debug seem that file kernel are too old for the ntfs3 build
Seems you are right, because I have faile with my script and kernel 5.4.65

Code: Select all

#!/bin/bash
# ntfs3-dkms driver
# NTFS read-write driver GPL implementation by Paragon Software.
# Current version works with NTFS (including v3.1), normal/compressed/sparse files and supports journal replaying.
# Slackware build script for broadcom-wl-dkms
# base on https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=ntfs3-dkms
# ported for Slackware by Blaze
# depends=(dkms)

PRGNAM=ntfs3
VERSION=${VERSION:-v6.0.0}
KERNEL=${KERNEL:-`uname -r`}
BUILD=${BUILD:-1}
TAG=${TAG:-bl}

if [ -z "$ARCH" ]; then
  case "$( uname -m )" in
    i?86) ARCH=i586 ;;
    arm*) ARCH=arm ;;
       *) ARCH=$( uname -m ) ;;
  esac
fi

CWD=$(pwd)
TMP=${TMP:-/tmp/SBo}
PKG=$TMP/package-$PRGNAM
OUTPUT=${OUTPUT:-/tmp}

if [ "$ARCH" = "i586" ]; then
  SLKCFLAGS="-O2 -march=i586 -mtune=i686"
  LIBDIRSUFFIX=""
elif [ "$ARCH" = "i686" ]; then
  SLKCFLAGS="-O2 -march=i686 -mtune=i686"
  LIBDIRSUFFIX=""
elif [ "$ARCH" = "x86_64" ]; then
  SLKCFLAGS="-O2 -fPIC"
  LIBDIRSUFFIX="64"
else
  SLKCFLAGS="-O2"
  LIBDIRSUFFIX=""
fi

set -e

rm -rf $PKG $TMP/$PRGNAM-$VERSION
mkdir -p $TMP $PKG/usr/src/ntfs3-${VERSION} $OUTPUT /usr/src/ntfs3-${VERSION} "${TMP}/${PRGNAM}-${VERSION}"

cd "${TMP}/${PRGNAM}-${VERSION}"

lynx --source https://aur.archlinux.org/cgit/aur.git/plain/dkms.conf?h=ntfs3-dkms > ${TMP}/${PRGNAM}-${VERSION}/dkms.conf
lynx --source https://aur.archlinux.org/cgit/aur.git/plain/Makefile.patch?h=ntfs3-dkms > ${TMP}/${PRGNAM}-${VERSION}/Makefile.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308074/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~1.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308079/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~2.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308077/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~3.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308073/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~4.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308069/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~5.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308070/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~6.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308078/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~7.patch
lynx --source https://lore.kernel.org/patchwork/patch/1308067/raw > ${TMP}/${PRGNAM}-${VERSION}/v6.0.0~8.patch

sed -i 's|$(KVERSION)|KERNEL|g' "${TMP}/${PRGNAM}-${VERSION}/Makefile.patch"
sed -i "s|KERNEL|${KERNEL}|g" "${TMP}/${PRGNAM}-${VERSION}/Makefile.patch"

for patch in "$VERSION~"*
do
    patch -p3 -N -i "$patch"
done
    patch -p1 -N -i "Makefile.patch"

cp -r "${TMP}/${PRGNAM}-${VERSION}" "/usr/src/"
install -Dm644 "${TMP}/${PRGNAM}-${VERSION}/dkms.conf" "/usr/src/ntfs3-${VERSION}/dkms.conf"

cd /usr/src/ntfs3-v6.0.0/
dkms install --no-depmod -m ntfs3 -v $VERSION -k `uname -r`
depmod `uname -r`

#cd "$OUTPUT/$PRGNAM-$VERSION-$KERNEL-$ARCH-$BUILD$TAG"
#/sbin/makepkg -l y -c n $OUTPUT/$PRGNAM-$VERSION-$KERNEL-$ARCH-$BUILD$TAG.${PKGTYPE:-txz}

Code: Select all

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...
KVERSION=5.4.65-porteus CONFIG_NTFS3_FS=m make KDIR=/lib/modules/5.4.65-porteus/build...(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.4.65-porteus (x86_64)
Consult /var/lib/dkms/ntfs3/v6.0.0/build/make.log for more information.

Code: Select all

# cat /var/lib/dkms/ntfs3/v6.0.0/build/make.log
DKMS make.log for ntfs3-v6.0.0 for kernel 5.4.65-porteus (x86_64)
Пн сен 21 18:17:23 MSK 2020
make -C /lib/modules/5.4.65-porteus/build M=/var/lib/dkms/ntfs3/v6.0.0/build modules
make[1]: вход в каталог «/usr/src/linux-5.4.65»
  CC [M]  /var/lib/dkms/ntfs3/v6.0.0/build/bitfunc.o
  CC [M]  /var/lib/dkms/ntfs3/v6.0.0/build/bitmap.o
/var/lib/dkms/ntfs3/v6.0.0/build/bitmap.c: В функции «wnd_rescan»:
/var/lib/dkms/ntfs3/v6.0.0/build/bitmap.c:558:4: ошибка: неявная декларация функции «page_cache_readahead_unbounded» [-Werror=implicit-function-declaration]
  558 |    page_cache_readahead_unbounded(mapping, NULL,
      |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: некоторые предупреждения считаются ошибками
make[2]: *** [scripts/Makefile.build:266: /var/lib/dkms/ntfs3/v6.0.0/build/bitmap.o] Ошибка 1
make[1]: *** [Makefile:1729: /var/lib/dkms/ntfs3/v6.0.0/build] Ошибка 2
make[1]: выход из каталога «/usr/src/linux-5.4.65»
make: *** [Makefile:14: all] Ошибка 2
The same error NTFS3 Kernel Module (Пост beny #78750)

But, you can use (tweak) my script for build ntfs3 module on a new kernel ;)
Linux 6.6.11-porteus #1 SMP PREEMPT_DYNAMIC Sun Jan 14 12:07:37 MSK 2024 x86_64 Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz GenuineIntel GNU/Linux
MS-7A12 » [AMD/ATI] Navi 23 [Radeon RX 6600] [1002:73ff] (rev c7) » Vengeance LPX 16GB DDR4 K2 3200MHz C16

beny
Full of knowledge
Full of knowledge
Posts: 2086
Joined: 02 Jan 2011, 11:33
Location: italy

NTFS3 Kernel Module

Post#17 by beny » 22 Sep 2020, 09:00

hi, seem we have to wait for aufs patch for the 5.8x kernel to build for porteus,with arch or current no problem with dkms or not but lts kernel is out of the build

User avatar
Blaze
DEV Team
DEV Team
Posts: 3869
Joined: 28 Dec 2010, 11:31
Distribution: ⟰ Porteus current ☯ all DEs ☯
Location: ☭ Russian Federation, Lipetsk region, Dankov
Contact:

NTFS3 Kernel Module

Post#18 by Blaze » 22 Sep 2020, 10:41

beny,

Code: Select all

+ dkms install --no-depmod -m ntfs3 -v v6.0.0 -k 5.7.19-porteus

Creating symlink /var/lib/dkms/ntfs3/v6.0.0/source ->
                 /usr/src/ntfs3-v6.0.0

DKMS: add completed.

Kernel preparation unnecessary for this kernel.  Skipping...

Building module:
cleaning build area...
KVERSION=5.7.19-porteus CONFIG_NTFS3_FS=m make KDIR=/lib/modules/5.7.19-porteus/build...(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.7.19-porteus (x86_64)
Consult /var/lib/dkms/ntfs3/v6.0.0/build/make.log for more information.

Code: Select all

# cat /var/lib/dkms/ntfs3/v6.0.0/build/make.log
DKMS make.log for ntfs3-v6.0.0 for kernel 5.7.19-porteus (x86_64)
Вт сен 22 13:39:47 MSK 2020
make -C /lib/modules/5.7.19-porteus/build M=/var/lib/dkms/ntfs3/v6.0.0/build modules
make[1]: вход в каталог «/usr/src/linux-5.7.19»
make[2]: Предупреждение: время изменения файла «include/config/auto.conf» находится в будущем (7945)
  CC [M]  /var/lib/dkms/ntfs3/v6.0.0/build/bitfunc.o
  CC [M]  /var/lib/dkms/ntfs3/v6.0.0/build/bitmap.o
/var/lib/dkms/ntfs3/v6.0.0/build/bitmap.c: В функции «wnd_rescan»:
/var/lib/dkms/ntfs3/v6.0.0/build/bitmap.c:558:4: ошибка: неявная декларация функции «page_cache_readahead_unbounded» [-Werror=implicit-function-declaration]
  558 |    page_cache_readahead_unbounded(mapping, NULL,
      |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: некоторые предупреждения считаются ошибками
make[2]: *** [scripts/Makefile.build:267: /var/lib/dkms/ntfs3/v6.0.0/build/bitmap.o] Ошибка 1
make[1]: *** [Makefile:1732: /var/lib/dkms/ntfs3/v6.0.0/build] Ошибка 2
make[1]: выход из каталога «/usr/src/linux-5.7.19»
make: *** [Makefile:14: all] Ошибка 2
Linux 6.6.11-porteus #1 SMP PREEMPT_DYNAMIC Sun Jan 14 12:07:37 MSK 2024 x86_64 Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz GenuineIntel GNU/Linux
MS-7A12 » [AMD/ATI] Navi 23 [Radeon RX 6600] [1002:73ff] (rev c7) » Vengeance LPX 16GB DDR4 K2 3200MHz C16

User avatar
babam
Warlord
Warlord
Posts: 526
Joined: 16 Nov 2016, 10:30
Distribution: Porteus 5.0rc3 Xfce K6.1.1
Location: Rainy city

NTFS3 Kernel Module

Post#19 by babam » 20 Oct 2020, 07:09

Here is NTFS3 for Porteus 5.0 RC2 (Kernel 5.4.57) https://github.com/LGA1150/ntfs3-oot

Code: Select all

# mount -t ntfs3 /dev/sdXX /mnt/sdXX -o noatime,nodiratime,suid,dev,exec,async,uid=1000,gid=100,umask=022,fmask=022,dmask=022,discard,nls=utf8,prealloc
Sorry, my English is bad.

User avatar
babam
Warlord
Warlord
Posts: 526
Joined: 16 Nov 2016, 10:30
Distribution: Porteus 5.0rc3 Xfce K6.1.1
Location: Rainy city

NTFS3 Kernel Module

Post#20 by babam » 26 Oct 2020, 05:56

Pros:
1. Very fast, much faster than ntfs-3g.
2. Prevents fragmentation when writing files, whereas ntfs-3g generates a lot of fragmentation.
3. Low in CPU and RAM usage.

Cons:
1. NTFS3 has not been tested for stability, reliability and security. NTFS-3G has been tested for stability and safety.
Sorry, my English is bad.

rych
Warlord
Warlord
Posts: 622
Joined: 04 Jan 2014, 04:27
Distribution: Porteus 5.0 x64 OpenBox
Location: NZ
Contact:

NTFS3 Kernel Module

Post#21 by rych » 17 Aug 2022, 13:12


User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 3924
Joined: 20 Mar 2012, 03:42
Distribution: v5.0-64bit
Location: australia
Contact:

NTFS3 Kernel Module

Post#22 by ncmprhnsbl » 17 Aug 2022, 23:06

rych wrote:
17 Aug 2022, 13:12
Does Porteus 5.0 by default use the new ntfs3 kernel driver or is it still FUSE ntfs-3g?
still FUSE ntfs-3g
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

burdi01
Shogun
Shogun
Posts: 201
Joined: 18 Aug 2013, 12:09
Distribution: Slackware PartedMagic Xubuntu
Location: The Netherlands

NTFS3 Kernel Module

Post#23 by burdi01 » 18 Aug 2022, 09:15

ncmprhnsbl wrote:
17 Aug 2022, 23:06
rych wrote:
17 Aug 2022, 13:12
Does Porteus 5.0 by default use the new ntfs3 kernel driver or is it still FUSE ntfs-3g?
still FUSE ntfs-3g
but you can use the ntfs3 driver: see https://www.linuxquestions.org/question ... 175702945/
:D

rych
Warlord
Warlord
Posts: 622
Joined: 04 Jan 2014, 04:27
Distribution: Porteus 5.0 x64 OpenBox
Location: NZ
Contact:

NTFS3 Kernel Module

Post#24 by rych » 23 Oct 2023, 08:38

ncmprhnsbl wrote:
17 Aug 2022, 23:06
still FUSE ntfs-3g
Maybe that's why it's slow. I find that the write speed off the host system's NTFS on NVMe SSD is twice slower than on the ext4 on an NVMe SSD USB 3 10Gbit thumbdrive.

Since the ntfs3 kernel driver was introduced in 5.15, there have been bugfixes and feature updates: for example in 5.19, in 6.2, in 6.4 and in 6.6 https://lore.kernel.org/ntfs3/

So, perhaps the early reports of (rare but shocking) NTFS file system corruption have now been addressed and the bugs fixed? I'll be trying to remount some of the less important, always backed-up NTFS partitions with ntfs3 driver instead of ntfs-3g and will test. Two questions: how do I it properly? Is it just

Code: Select all

umount /dev/sda4
mount -tntfs3 /dev/sda4 /mnt
Second question/request to ncmprhnsbl: thanks for updating us to Porteus 5.01 with the kernel 6.5.5, could you offer kernel updates as Updates to Porteus like you do with modules? I know it involves rebuilding Nvidia driver etc. but perhaps when it hits a 6.6 and other milestones? It's because the ntfs3 bugfixing is still in progress and some are checked in for 6.6 and so on.

rych
Warlord
Warlord
Posts: 622
Joined: 04 Jan 2014, 04:27
Distribution: Porteus 5.0 x64 OpenBox
Location: NZ
Contact:

NTFS3 Kernel Module

Post#25 by rych » 30 Oct 2023, 09:14

burdi01 wrote:
18 Aug 2022, 09:15
you can use the ntfs3 driver
How? Its module doesn't seem to be loaded by default and can't be loaded:

Code: Select all

root@porteus:~# lsmod | grep ntfs
root@porteus:~# modprobe ntfs3
root@porteus:~# lsmod | grep ntfs
root@porteus:~# 
root@porteus:/mnt/sdb3# systool -v -m ntfs3
Error opening module ntfs3
Perhaps it's somehow active anyway:

Code: Select all

root@porteus:/mnt/sdb3# modinfo ntfs3
name:           ntfs3
filename:       (builtin)
alias:          fs-ntfs3
author:         Konstantin Komarov
compression:    Read-only lzx/xpress compression included
behaviour:      Enabled Linux POSIX ACLs support
description:    ntfs3 read/write filesystem
license:        GPL
file:           fs/ntfs3/ntfs3
There are ntfs3 messages in:

Code: Select all

root@porteus:~# dmesg | grep ntfs3
[    0.286414] ntfs3: Max link count 4000
[    0.286415] ntfs3: Enabled Linux POSIX ACLs support
[    0.286416] ntfs3: Read-only LZX/Xpress compression included
[ 2410.517027] ntfs3: sdb3: It is recommened to use chkdsk.
[ 2412.925950] ntfs3: sdb3: volume is dirty and "force" flag is not set!
It's a known file system according to

Code: Select all

root@porteus:/mnt/sdb3# cat /proc/filesystems | grep ntfs
	ntfs
	ntfs3
If anyone manages to mount an NTFS with ntfs3, could you teach me. For now I'm getting these results:

Code: Select all

root@porteus:/mnt/sdb3# mount -fv -tntfs3 /dev/sdb3
mount: /dev/sdb3 mounted on /mnt/sdb3.
root@porteus:/mnt/sdb3# umount -v /dev/sdb3
umount: /dev/sdb3: not mounted.
root@porteus:/mnt/sdb3# mount -v -tntfs3 /dev/sdb3
mount: /mnt/sdb3: wrong fs type, bad option, bad superblock on /dev/sdb3, missing codepage or helper program, or other error.
I feel like there are rumors of ntfs3 on my system but the module itself is not actually included with the kernel (Porteus 5.01)?!

rych
Warlord
Warlord
Posts: 622
Joined: 04 Jan 2014, 04:27
Distribution: Porteus 5.0 x64 OpenBox
Location: NZ
Contact:

NTFS3 Kernel Module

Post#26 by rych » 30 Oct 2023, 10:14

OK, after

Code: Select all

ntfsfix --clear-dirty /dev/sdb3
this works:

Code: Select all

mount -t ntfs3 /dev/sdb3
findmnt | grep ntfs
└─/mnt/sdb3  /dev/sdb3 ntfs3 rw,nosuid,nodev,noatime,nodiratime,uid=0,gid=0,iocharset=utf8

fulalas
DEV Team
DEV Team
Posts: 2050
Joined: 26 Oct 2016, 15:34
Distribution: Porteus
Location: Brazil

NTFS3 Kernel Module

Post#27 by fulalas » 30 Oct 2023, 10:53

rych wrote:
30 Oct 2023, 10:14
OK, after

Code: Select all

ntfsfix --clear-dirty /dev/sdb3
this works:
Nice! Now, the big question: what about performance? :D

rych
Warlord
Warlord
Posts: 622
Joined: 04 Jan 2014, 04:27
Distribution: Porteus 5.0 x64 OpenBox
Location: NZ
Contact:

NTFS3 Kernel Module

Post#28 by rych » 30 Oct 2023, 11:01

fulalas wrote:
30 Oct 2023, 10:53
Now, the big question: what about performance?
Looks good already: Booting from NVMe USB enclosure (Post by rych #96861). Will try on larger faster disks soon -- a bit afraid, so will backup first

rych
Warlord
Warlord
Posts: 622
Joined: 04 Jan 2014, 04:27
Distribution: Porteus 5.0 x64 OpenBox
Location: NZ
Contact:

NTFS3 Kernel Module

Post#29 by rych » 04 Nov 2023, 09:36

ncmprhnsbl, Now that I've tested it for a while with no stability/integrity issues whatsoever, could you help me replace ntfs-3g with ntfs3 in mounting all NTFS partitions at boot please? I think I need to modify these 2 lines in the finit:

Code: Select all

ntfs-3g /dev/$1 /mnt/$1 -o $MOPT,big_writes 2>/dev/null || rmdir /mnt/$1
...
ntfs-3g /dev/$x /mnt/$x -o $MOPT,big_writes 2>/dev/null || { sed -i "/$x /d" /etc/fstab; rmdir /mnt/$x; }
Firstly: why there are two occurrences of that mount command and which one applies?

Secondly, how do I properly rewrite it using ntfs3?

At the moment, I re-mount a posteriori some partitions that I still can, that are not "busy", inheriting the same mount options and only changing the driver, with e.g.,

Code: Select all

umount /dev/sdb3
mount -t ntfs3 /dev/sdb3
... and it works: options and the mount point are read from fstab I guess. I could add some ntfs3 specific options too...

But I'd really like for all my NTFS partitions to be mounted from the start with ntfs3 if possible.

rych
Warlord
Warlord
Posts: 622
Joined: 04 Jan 2014, 04:27
Distribution: Porteus 5.0 x64 OpenBox
Location: NZ
Contact:

NTFS3 Kernel Module

Post#30 by rych » 05 Nov 2023, 09:57

I suggest replacing

Code: Select all

ntfs-3g /dev/$1 /mnt/$1 -o $MOPT,big_writes 2>/dev/null || rmdir /mnt/$1
with

Code: Select all

mount -n -t ntfs3 /dev/$1 /mnt/$1 -o $MOPT 2>/dev/null || rmdir /mnt/$1
Is that okay? Strange that that errors are not diplayed. There could be an important error in case of the ntfs3 mounting: for example failing due to the dirty bit. Instead the commands above remove /mnt/$1 in case of error -- is that necessary?

Anyway, treading carefully, can I just replace everywhere
ntfs-3g /dev/$1 /mnt/$1 -o $MOPT,big_writes
-->
mount -n -t ntfs3 /dev/$1 /mnt/$1 -o $MOPT

(Also an interesting option should be -n is used, why?)


Another relevant place is in the linuxrc file, and I suggest this change:

Code: Select all

sed -i 's/ ntfs / ntfs-3g /g' /etc/fstab
-->

Code: Select all

sed -i 's/ ntfs / ntfs3 /g' /etc/fstab
(this command searches-and-replaces in the /etc/fstab)

Post Reply