Thanks, pizzar0! Please understand that I very much appreciate your comments and contributions -- my only goal here is to understand all of this better, to improve my own knowledge and to share what I learn with the rest of the community. You have been a great help towards this goal.
To answer your single question, I think that it would be of moderate importantance to Porteus' ultimate success to find and select the optimum file system for the intended use of this specific distribution (this is in my opinion only). Why do I think it is of only moderate importance, and why do I think it should not necessarily be implemented as a default during installation? Because at the end of the day, if we're talking about the native filesystem for a flash drive on which Porteus is installed, it is of only moderate importance, and highly open to user customization without much risk. This is because the majority of the system is constructed from read-only modules, so the internals of the initrd and the live filesystem (not to mention the content thereof) have much more to do with the actual useability, performance and end-user experience of Porteus. Furthermore, many users will never get to the point where they will want to repartition their drive away from windows compatibility, so we ought to remain flexible enough for users to customize their choices in regards to their native filesystem of choice (to this point, let me add that read speed ultimately will be of more importance to most users than write speed, because Porteus reads from the disk far more often than writes to it; XFS is in a virtual tie with most other filesystems in this regard, as read speed seems to be much more a function of hardware capabilities than filesystem attributes). The question that naturally arises then is, "If this is not really that important, why the hell is Ahau spending so much god damn time on it?" I don't have a good answer myself (I wish I did!), other than I'm curious, and I want to find the answer, and the more time I spend on it without a resolution, the more it bugs me. Also, I want to share my findings with the rest of the community, so that they don't have to spend a month doing what I'm doing, in order to answer a complex question of minor importance.
Thank you for clarifying about the barriers vs no barriers comments by the designers
See the end of my post for more on this.
Also, thank you for explaining your experiences with slax and XFS -- were you using XFS as the native filesystem on your block devices, running it as the default filesystem inside the initrd and /dev/root (i.e., changing the default initrd/linuxrc, which I believe are set to create virtual ext2 systems for both slax and porteus v1.0), or both? I ask the question because expert configuration can be applied to the live filesystem on the administration end (in fanthom's latest development release, the initrd was updated to be an ext4 image instead of ext2, so now would be the time to dig into this question and resolve which is really best), but this becomes more problematic on block devices when we have to factor in various device specifications and user experience/knowledge levels (I think we've sufficiently beaten that dead horse to the point where we agree on this).
I agree that we are all entitled to our own opinions, and not our own facts -- I have seen a lot of opinions about flash drives, alignment, geometry, and filesystems, but not a lot of reliable facts on the internet -- that is why I have set out to generate some testable conclusions backed by reproducible data. I should have worded my statement about "the manufacturer's preferences" more carefully -- my intent was not to discard the opinion of the manufacturers. Rather, I wanted to reinforce the need to understand the reasoning behind why they made their decisions and in what instances they are applying them, so that I could evaluate (to the best of my ability) whether or not those same conclusions are valid for our uses. If, for example, they are using XFS because they are primarily working on enterprise class servers with huge files, and are backing up their logs to USB devices, this is a much different situation than running a linux-live distribution on a flash drive. You've explained that they use it in many instances, but I would like to know if it was selected for reasons of performance, reliability, compatibility with other hardware, etc. (note that this is a hypothetical statement, and I'm not asking for or expecting some kind of detailed report from you -- but if you have some more links that point to this, I'd gladly read them, and I'll do some more research on my own)
I don't have any illusions that OSS is naturally better than commercially developed products, nor would I imagine that SGI was out to trick anybody. I entered this experiment with no knowledge of the origins of XFS or it's speed and reliability; my conclusions have been based on the data produced in my (admittedly simple) experiments and the information I have been able to gather elsewhere. I have no vested interests or axe to grind against XFS or big corporations (I work for one myself).
I couldn't agree more with you that the characteristics of the device is of far more importance than which filesystem you choose, when discussing performance issues. Yes, experts can and will tweak devices to obtain awesome performance (as you illustrate), but by and large, better and newer hardware will outperform older and crappier hardware, regardless of how you format the drive.
What I'm after here is a simple answer to a complex question. If a (relatively) new user asks, "what filesystem should I put on my flashdrive, for use with Porteus", I want to be able to provide a somewhat generic, easy to implement response, such as, "In my experience, XXX filesystem provides the best performance, but XXX filesystem is somewhat more reliable in cases of power failure, but in all cases, you're better off using XXX or YYY than using FAT with a save.dat"
Thank you as well for your discussion of the Corsair Voyager (333MB--sweet!) and your experience with Slax installed on XFS -- it's useful to know how reliable XFS has been on so many systems! This is one area where it is especially difficult for me to gather data myself. I can force a number of crashes and document the results, but I simply don't have the time necessary to gather long term data on reliability in normal use for each filesystem.
And finally, back to the external log and barriers discussion: your first post in this thread included your mkfs and mount options for the Corsair Voyager drive, as follows:
mkfs.xfs -f -l logdev=/dev/loop(X-1),size=128m,lazy-count=1 -d agcount=32
mount -t xfs -o noatime,nodiratime,logdev=/dev/loop9,logbufs=8 /dev/loop10 /VoyTest
I have tried to recreate this on my end, but have repeatedly failed to establish a loop device as the external log device. I've also been able to find little to no information on the web about how to set this up (primarily, I find information about putting the external log on a block device). Could you please provide some more detail on this process, so that I can recreate it and incorporate the results into my analysis?
I assume that /dev/loop(X-1) refers to a loop device that is one lower than the loop device you've established for your filesystem (which itself resides on a loop-backed image file). If I do nothing to establish /dev/loop18, for example, mkfs.xfs will not accept it as the external log device. If I create an image file and mount it on /dev/loop18, it fails again (but tells me that the specified device contains a formatted filesystem). To clarify, I am trying to create an actual partition on my flashdrive (e.g. /dev/sdb2), not a loop-backed image file, but I want the log to go someplace else. I guess my ultimate question is this: if you really are putting your log on a loop device that isn't tied to a separate block device, where does the log go, in case of a power failure? And, if you are placing the log on a separate block device, it seems that you would be limited to only booting Porteus on that one machine (or I suppose you could put the log on another flash drive, and use both in concert for each machine).
Using a higher agcount seems to slow down my drive (I think it's probably a dual channel drive, and mkfs.xfs defaults to 4, which would be the appropriate amount per your initial instructions). I realize how this would be beneficial if I had a quad channel (or higher) device -- your Corsaire drive seems to have 16 channels -- egads, that's quick!
Again, many thanks for your participation and help!
Posted after 51 minute 47 seconds:
Re: Finding the best filesystem (for USB flash drive installs)
Some interesting, if outdated info on filesystem reliability, from Toshiba, recorded here for posterity:
http://elinux.org/images/e/e2/Evaluatio ... ayashi.pdf