Portable Dogdy Filesystems in Userland (hack)

(alias podfuk)

Why came into being podfuk ?

We have to suppose we have something (in our case filename.tar) and we would like use it as filesystem. (Instead of filename.tar "something" can be ftp server or Macintosh diskette etc.) Which opportunities do you have?

With Podfuk we can relatively, easily add filesystems which don't need implementation in the kernel.

Disadvantages

Podfuk uses NFS which is a big advantage but also big disadvantage. Protocol NFS is broken by design. Still, NFS works quite well in real applications. NFSv2 is slow in some operations for examples listing of files (ls in directory /overlay/dev lasts for one second)

Universal NFS server was built as singlethreaded. It means in one time it does only one task (and expects every request to finish pretty soon). That is unfortunately for a lot of filesystems unacceptable - reading for and of files connected by ftp can last one hour. This could be solved. What is worse, Linux implementation of NFS client does not expect NFS server to behave like that. For example if NFS server is not responding to readdir, linux client will not send another readdir.

Right way would be to correct NFS client in Linux and then to make server multithreaded.

In another words Podfuk doesn't support ftp filesystem and mc's network filesystem.

Midnight Commander

Midnight commander is Norton Commander clone, developed under GPL by group of people around Miguel de Icaza. One of its nice features is its portability, other feature is mc's supportfor virtual file systems (VFS). Midnight Commander has VFS component which can easily works with tar, ftp, zip, rar, deb, rmp etc. as filesystems. VFS component exports to public functions mc_open (), mc_read (), mc_close (), mc_unlink (), ... The difference between these functions and their system counterpairs: they can work with virtual files. So

handle = mc_open( "soubor.tar#utar/README", O_RDONLY ); 
mc_read( handle, buffer, 10240 );

really reads first 10K from file README packed into archive filename.tar.

In past VFS was undivided component of Midnight Commander. Now this is stated but now it is possible VFS to build VFS separately. The product of compilation is libvfs.so which can be used by anyone. (Actually, anyone working under GPL. I'm hoping to change that.)

Next step is easy. Our NFS server must use for its functions library VFS. We must replace some occurrences of functions xxx with functions mc_xxx.

How to use

Path look like from the view of podfuk as /path.../filename#operation[/path2 in virtual tree].

Operation is name of one VFS (for example #utar, #ugz), path2 is the path inside tar archive (#utar) or path inside ftp server (#ftp filesystem). There is no path2 for things like #ugz.

Filename is name of the current file/directory or / if such thing has no sense. (so directory of midnight commander at sunsite will be /#ftp:sunsite.ms.mff.cuni.cz/pub/GNU/mc/).

For showing to the virtual files Podfuk must know about access to that files. That means if you have Podfuk mounted in directory /overlay a you want to read the file Readme from /tmp/patch.tar you must use /overlay/tmp/patch.tar#utar/README (if you don't begin with /overlay, kernel will have no reason to ask our NFS-server).

Kernel hack

Adding manually /overlay to the path is not comfortable. The solution of this problem is kernel hack. This hack is not portable. It functions that if applications doesn't found the path, kernel will try path beginning with /overlay.

More informations

Another information is at the home page of Podfuk.


This text was originally created by Pavel Machek (in czech), then translated to english by Ondrej Mastalka, then heavilly modified and translated back to html by Pavel Machek. Sorry for bad english (feel free to submit wording corrections).

Pavel Machek
pavel@atrey.karlin.mff.cuni.cz