Linux nowadays comes with integrated support for FUSE which allows all kinds of interesting user level file systems. One of the most interesting might well be avfs which allows you to treat any archive (like zip, tar, etc) as if it was a directory.
This is particularly handy, if you want to browse archives through apache. In our test automation environment we generate and receive many megabytes of log files every day. Log files from several clients are zipped together and send to our "visulization center". We extract some common information out of the log files which is displayed in a web-app and we keep the log files for reference (in their compressed format). On the test results page we have links that link right into the log file archives. Obviously this "zip browsing" could be implemented in the web-app itself, but why bother apache with avfs has all you need.
The only problem is, that for the current stable release of debian (etch) avfs is not available and can not be compiled against stable versions of libfuse2/libfuse-dev. However there is a simple solution to this, you can get updated fuse packages from backports.org.
Before you follow this guide: Make sure that you really (still) need etch.
This howto does not strive to be a complete help guide! Please read the information available on the backports.org page very carefully.
You may want to start with a system update, in other words
apt-get update apt-get install kernel-image-2.6-686 apt-get dist-upgrade update-grub sync reboot
You need to add etch-backports to your /etc/apt/sources.lst. The following is a very short description, for details please see backports.org. Add the following lines
deb http://www.backports.org/debian etch-backports main contrib non-free deb-src http://www.backports.org/debian etch-backports main contrib non-freeYou also need to get the backports pgp keys
wget -O - http://backports.org/debian/archive.key | apt-key add -Once this is done update your package lists
apt-get update
The packages we need for this to work are:
apt-get -t etch-backports install libfuse-dev apt-get -t etch-backports install libfuse2 apt-get -t etch-backports install fuse-utils
We will get the lenny (testing) packages of avfs which compile fine against the fuse version from backports. You may want to check with debian if there are later versions available by now and update the following commands accordingly.
curl -O http://ftp.de.debian.org/debian/pool/main/a/avfs/avfs_0.9.8.orig.tar.gz curl -O http://ftp.de.debian.org/debian/pool/main/a/avfs/avfs_0.9.8-6.diff.gz tar -zxf avfs_0.9.8.orig.tar.gz gunzip avfs_0.9.8-6.diff.gz mv avfs-0.9.8.orig avfs-0.9.8-6 && cd avfs-0.9.8-6 && patch < ../avfs_0.9.8-6.diff
Let's compile and install.
./configure --enable-fuse make clean all make installIf the configure shows you that it doesn't like your version of fuse (which it shouldnt with the backports packages), you can try the option --disable-fuse-test.
avfs works on a per user basis. If you just want to use it as a regular user, you need to create a .avfs subdirectory in your homedir and start the daemon (each user on its own). You also need to be able to access /dev/fuse which is possible if you are in group fuse.
As user root you will have to edit /etc/group or use
usermod -G fuse -a <user>to add the users that should be able to use avfs.
Each user that wants to use acfs has to execute:
mkdir ~/.avfs mountavfs
Well that's it for avfs. You are still reading, so it seems, things are going well so far. Let's give a quick try.
(cd /tmp && echo "Hide me up" > bla && tar -zcf bla.tar.gz bla && rm bla) cd ~/.avfs/tmp/bla.tar.gz#/ ls cat bla
Combining avfs and apache can be very useful if you want or need to store large amounts of data in a compressed format and still want to be able to browse it.
We want to run avfsd for the "apache user" and we want it to mount to a "global" mountpoint.
sudo -u www-data avfsd -o modules=subdir -o subdir=/projects/fasthosting/ -o rellinks /var/www/archiveroot/will mount the directory /projects/fasthosting with avfs at /var/www/archiveroot accessible to the user www-data. It will also transform any absolute links to relative links so that you stay within /var/www/archiveroot.
You may want to try option -f -d to get an idea what is going on in case things don't work.
I will just provide an example here which works in our particular case - all of our archive files are .zip files, but you should get the idea. The trick is to change the URL such that you add a # after the zip filename (which is a bit tricky because # has a special meaning in urls...)
Alias "/browse" "/var/www/archiveroot"
<Directory /var/www/archiveroot>
Allow from All
Options Indexes SymLinksIfOwnerMatch
AllowOverride None
RewriteEngine On
RewriteBase /browse
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^(.*?.zip)$ $1/BROWSE/ [R,L]
RewriteRule ^(.*)/BROWSE/(.*)$ $1\%23/$2 [NS,L,NE]
</Directory>
This should work out of the box, give it a try - don't forget to update the RewriteBase if you change the Alias directive.
If you want to use this in a productive environment you will need to start avfsd at startup. Here is my init.d script based on the debian skeleton, yours may need some adaptation.
#! /bin/bash ### BEGIN INIT INFO # Provides: fast_avfs # Required-Start: $local_fs $remote_fs # Required-Stop: $local_fs $remote_fs # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start avfsd for user www-data # Description: Starter for the fuse plugin avfs which allows browsing of zip files. # Used by the fasthosting apache configurations to view inside uploaded zip file. ### END INIT INFO # Author: Joachim Buechse# Do NOT "set -e" USER=www-data GROUP=www-data MOUNT_POINT=/var/www/archiveroot/ BASE_DIR=/projects/fasthosting/ # PATH should only include /usr/* if it runs after the mountnfs.sh script #PATH=/sbin:/usr/sbin:/bin:/usr/bin DESC="archive browser for fasthosting (user www-data)" NAME=fast_avfs DAEMON=/usr/bin/avfsd DAEMON_ARGS="-o modules=subdir -o subdir=$BASE_DIR -o rellinks $MOUNT_POINT" PIDFILE=/var/run/$NAME.pid SCRIPTNAME=/etc/init.d/$NAME # Exit if the package is not installed [ -x "$DAEMON" ] || exit 0 # Read configuration variable file if it is present #[ -r /etc/default/$NAME ] && . /etc/default/$NAME # Load the VERBOSE setting and other rcS variables . /lib/init/vars.sh # Define LSB log_* functions. # Depend on lsb-base (>= 3.0-6) to ensure that this file is present. . /lib/lsb/init-functions # # Function that starts the daemon/service # do_start() { # Return # 0 if daemon has been started # 1 if daemon was already running # 2 if daemon could not be started start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \ || return 1 start-stop-daemon --start --quiet --make-pidfile --pidfile $PIDFILE --background --chuid $USER:$GROUP --exec $DAEMON -- \ -f $DAEMON_ARGS \ || return 2 # we start avfsd in foreground mode a create PIDFILE with the correct PID } # # Function that stops the daemon/service # do_stop() { # Return # 0 if daemon has been stopped # 1 if daemon was already stopped # 2 if daemon could not be stopped # other if a failure occurred # check if the mountpoint exists, if not (directory) we are done [ -d $MOUNT_POINT ] && return 1 #echo "unmounting with fusermount" /usr/bin/fusermount -u $MOUNT_POINT RETVAL="$?" [ "$RETVAL" = 0 ] && [ -d $MOUNT_POINT ] && rm -f $PIDFILE && return 0 #echo "killing process" start-stop-daemon --stop --quiet --retry=TERM/30 --pidfile $PIDFILE --name $NAME RETVAL="$?" [ "$RETVAL" = 2 ] && return 2 rm -f $PIDFILE return "$RETVAL" } case "$1" in start) [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME" do_start case "$?" in 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; esac ;; stop) [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME" do_stop case "$?" in 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; esac ;; restart|force-reload) # # If the "reload" option is implemented then remove the # 'force-reload' alias # log_daemon_msg "Restarting $DESC" "$NAME" do_stop case "$?" in 0|1) do_start case "$?" in 0) log_end_msg 0 ;; 1) log_end_msg 1 ;; # Old process is still running *) log_end_msg 1 ;; # Failed to start esac ;; *) # Failed to stop log_end_msg 1 ;; esac ;; *) echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2 exit 3 ;; esac
If this works for you, don't forget to activate it. I'm starting and stopping it at the same time as apache which is S91 and K09 on our installation.
update-rc.d fast_avfs defaults 91 09
LINKS:
(c) joachim(et)buechse.ch. This description is in the public domain.