NTFS3 Kernel Module

[tome]

I decided to use ntfs3 at start of porteus, here --> https://www.dropbox.com/sh/gdzuwmrramlv ... kfYbZMrUra are:
1. initrd-tm3.xz that use ntfs3 at boot
2. udev-fstab-update for ntfs3 - with this new ntfs mounts after boot would be also done by ntfs3 driver instead of ntfs-3g (without it ntfs3 at boot only if auto/automount cheatcode)
3. UFSD-utils-from-Root-Mounter-for-Android-by-Paragon-....xzm - some utils like fsck for ntfs, mount_ufsd_fuse - for example to test its speed

Changes to booting of Porteus:
4. If fsck cheatcode is used then ntfs partitions are check with paragon chkntfs -a -f
5. and be pressing any letter(key) while blue background is displayed you can stop booting to read easy output of fsck commands
6. this initrd includes also cheatcodes from this topic:
New initrd.xz and cheatcodes (boot parameters) for Porteus

7. ntfs-3g provide also mount.lowntfs-3g - it uses a lower-level FUSE API, and it seems it's faster than mount.ntfs-3g (amazing?, as fast as the ext4!?)
8. Microsoft NTFS by Tuxera (not ntfs-3g) is a commercial-grade file system implementation that is 6X faster than open-source alternatives.

ad. 1) probably uid=1000,gid=100 would be useful option at boot, now is's omitted

[fulalas]

@tome, have you managed to mount a NTFS folder (using ntfs3 driver) into the union without making the whole folder in union read-only?

I know we can do that using folders from traditional POSIX partitions, like ext4, but I'm failing to so using NTFS. The result is either read-only in union (which is useless) or writable but then it has whiteouts, changes, chaos.

[rych]

I'm on ntfs3 and unfortunately the following occasionally happens to this day. When I occasionally have a system freeze (I think due to running out of RAM having no swap partition/file set up) and can only restart by long-pressing the power-button, there are NTFS partition problems. Often it's a matter of checking it under Windows and the chkdsk /f finds no errors. But sometimes there are recovered chunks of files that were open(?) at the moment of powering off.

This may be because: "the NTFS3 driver still does not fully support NTFS journaling, i.e. if you lose power to a mounted disk, there's a good chance you'll have corruption" and "a power outage/unclean mount was likely to corrupt something, leaving inconsistencies".

Indeed I remember ntfs-3g was more forgiving and gracious in such situations. I'll keep using ntfs3 for now as it gives me the speed of ext4 but perhaps lowntfs-3g is a better alterative, I'll try to find out.

[fulalas]

@rych, which kernel are you using?

Have you managed to recover the corrupted files? Have you tried using chkufsd posted by tome (inside initrd)?

[rych]

which kernel are you using?

Code: Select all

root@porteus:~# uname -r
6.6.20-porteus

Have you managed to recover the corrupted files?

I didn't need them, they are synched from USB in my scenario anyway. Still it's disturbing. Windows (?) when repairing (?) the drive created some folders C:\found.001 or something filled with lost of useless chunks. Anyway, all is now fixable with

Code: Select all

chkntfs /dev/disk/by-label/Windows -f

Although I took it from his UFSD-utils-from-Root-Mounter-for-Android-by-Paragon-2017-2024-04-15_tm.xzm and not the initrd...

Thank you tome, and thanks jjr for communicating the chkntfs (chkufsd) to us. Not sure if it should go inside initrd, but I'll keep it as the above module at all times when using ntfs3. Ntfs3 will occasionally corrupt things, but chkntfs fixes them beautifully. Good partnership

[fulalas]

@rych, maybe you should try a newer kernel because I heard many fixes related to ntfs3 have been merged over the last months.

I have been using ntfs3 on kernel 6.9 for 2 days and so far so good. Unfortunately I can't mount NTFS folders into union without having the issues I mentioned here.

EDIT: one thing that is simply amazing regarding ntfs3 is that it allows compatibility with POSIX, meaning we can finally have permission settings.

[rych]

Thanks fulalas, will do for sure, but maybe after it matures a bit further? You've reported this:

Kernel 6.9 is out. However, I noticed a 10% performance drop

[fulalas]

After a week testing ntfs3, my opinion is that it's great only when the machine behaves 100% stable all the time:
1. it's considerably faster than ntfs-3g
2. it provides some POSIX compatibility
3. it's reliable (again, when the machine is 100% stable)
4. it's native to the kernel, not requiring any additional package

However, at the minimum disturbance in the stability of the machine things might get messy:
1. forced reboots, including hitting ctrl+alt+del in a tty, might corrupt files or even the whole partition to the point that sometimes it can't be recognized by the system
2. annoyingly, it doesn't provide an official tool to fix any of that. Using ntfs-3g fix utility doesn't solve these problems, requiring either Windows tools or chkufsd (which is not native and it's not even open source)
3. POSIX compatibility doesn't look 100% at the moment (kernel 6.9.1), and AUFS requires the parameter 'icex' to ignore invalid permissions when a NTFS folder/partition is mounted into union during boot time, otherwise the mounted folders will become read-only (more here)
4. the migration from ntfs-3g to ntfs3 (and vice-versa) will break absolutely all symlinks

TL;DR: it's not ready for production, only for enthusiasts.

[rych]

fulalas Sadly I have to agree. Although, under the forced reboots etc., both ntfs-3g and even ext4 file systems may get damaged and need fixing on the next reboot using their native tools. The newly discovered chkntfs seem to fix the corrupted ntfs3-mounted partitions well for now. I guess we want to see future kernel development regarding: "the NTFS3 driver still does not fully support NTFS journaling".

[rych]

Currently each time I write on my NTFS USB disk mounted with ntfs3, it's marked dirty!? GParted's warning reads:

Forced to continue.
$MFTMirr does not match $MFT (record 3).
Failed to mount '/dev/sdb4': Input/output error
NTFS is inconsistent. Run chkdsk /f on Windows then reboot it TWICE!
The usage of the /f parameter is very IMPORTANT! No modification was
made to NTFS by this software.

I don't bother to plug it into a Windows machine just for fixing stupid errors though, instead I fix(?) it daily with:

Code: Select all

root@porteus:~# uname -r
6.8.10-porteus

root@porteus:~# chkntfs /dev/disk/by-label/WorkUSB -f
Checking Volume /dev/disk/by-label/WorkUSB...
Type of the filesystem is NTFS.
Volume label is: WorkUSB.
Volume is marked as dirty.
Verifying 386304 records...
Verifying 95627 file(s) with EAs...
Verifying 24461 folders...
Minor inconsistencies detected on the volume. 1 file contains incorrect links count.
Verifying files security...
$MFTMirr does not match $MFT at offset 0xc30 from 0x1000.
Correcting $MFTMirr data.
$UpCase file is formatted for use in Windows 7 and later versions.
Checking volume bitmap (27503 kBytes)
    859.46 GB total disk space.
    197.55 GB in 362214 files.
    133792 KB in 24463 directories.
    616740 KB in use by the system.
        64 MB occupied by the log/journal file.
         4 KB in each allocation unit.
 225303039 total allocation units on volume (859.46 GB).
 173361005 allocation units available on volume (661.32 GB).
The volume /dev/disk/by-label/WorkUSB has been repaired successfully.

I don't like this of course. But I'll wait for an official Porteus 6.9 kernel update and report then.

[burdi01]

Actually I saw this problem with my own 6.9 kernels on Slackware Current too, though not as freqently as rych reports. I fell back to the ntfs-3g driver for everyday's interactive work, but for copying whole 300+ GB filesystems the ntfs-3g driver grinds to a halt, so for that function I use the ntfs3 driver.

[tome]

@tome, have you managed to mount a NTFS folder (using ntfs3 driver) into the union without making the whole folder in union read-only?

I know we can do that using folders from traditional POSIX partitions, like ext4, but I'm failing to so using NTFS. The result is either read-only in union (which is useless) or writable but then it has whiteouts, changes, chaos.

Now I booted with changes on ntfs3 partition and everything is correct, I have .wh..wh.orph, .wh..wh.plnk, .wh..wh.aufs , but apparently that's the way it should be. Test in terminal with ovarlayfs and ntfs3 is correct also. Kernel 5.18.8.

[rych]

$MFTMirr does not match $MFT

That error is (permanently?) gone after

Code: Select all

root@porteus:~# ntfsfix /dev/sda3
Mounting volume... OK
Processing of $MFT and $MFTMirr completed successfully.
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/sda3 was processed successfully.
root@porteus:~# 

[rych]

That error is (permanently?) gone after

Nope: after reboot SpaceFM refuses to mount it (Gparted can still mount it). I think ntfsfix is a utility from ntfs-3g. Now in the ntfs3 realm we run this command to fix:

Code: Select all

chkntfs /dev/sda3 -f
Checking Volume /dev/sda3...
Type of the filesystem is NTFS.
Volume is marked as dirty.
...
The volume /dev/sda3 appears to be OK.

Now SPaceFM mounts it alright. But when I write anything on it, GParted displays the same error as before:

Forced to continue.
$MFTMirr does not match $MFT (record 3).

So we've done the full circle. I'll try the same under PorteuX which have a newer kernel driver for ntfs3 and shall report here.

UPDATE: same under PorteuX 2.1

However, in conclusion, the error in the GParted doesn't manifest itself anywhere else and goes away after ntfs3-remounting. Also, GParted mounts NTFS differently from SpaceFM for example. Maybe even as ntfs-3g rather!? I won't be checking this anymore. This is borderline flooding. Basically, ntfs3 has been stable for my usage, and I let GParted complain.