Search:  
Gentoo Wiki

Apache_chroot:_the_mod_security_way

Apache2 Series


Main


Modules


Addons & Tunnels


Tips


Configuring


Other

edit

Contents

The goal

A chroot can be used as a pre-emptive way of containing a security breach by preventing a would-be attacker from doing any damage or probing the host system with a compromised program (http://en.wikipedia.org/wiki/Chroot).

About security, chrooting apache should it be better, anyway assume at your own risk any problem arising from this howto.

This howto has two principal purposes:

  1. Chrooting apache should be difficult, because it's a complex program with a lot of dependencies.
    We'll show you how mod_security gives an easy way to install apache in the usual location, but running it inside the chroot.
  2. Running in chroot, apache may need some feature which is not available in the jail.
    We'll show you how to populate your apache jail, adding some of these features.


Required in advance

You will need to have a functional apache setup, Apache setup and installation is out of the scope for chrooting apache. This article was tested with Apache 2.2.6, MySql 5.0 and ModSecurity 2.1.2.

Please install mod_security at this point if you have not done so.

# emerge -av mod_security

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    ] net-www/mod_security-2.1.2  USE="-doc"
Note: mod_security and mod_chroot both claim to chroot apache with no additional files needed. This is true if all you want to serve up is some circa 1990 web page with static html pages and images. In the real world if you need php, mysql, python, perl, ruby, mail, dns or anything else you need to setup additional files.

Building the jail

First we need to select a location for the jail. In this example I will be using /wwwjail. You can set this up where ever you like, even on a SAN.

# mkdir /wwwjail

Now there are two ways to prepare the actual chroot directory structure. The first is to create the structure manually and the second is to use the jail package. If you dont mind creating a few folders and copying some files the manual method should be just fine. However if you are lazy and/or need to jail some users that maybe accessing the jail via ssh or scp, you should use jail.


Building the chroot manually

Please skip this section if you plan on using jail to build the chroot.

This example we will create our chroot directory structure of two example sites called site1 and site2. These should be replaced by your hosts and repeated for each of them.

# cd /wwwjail
# ln -s . wwwjail

# mkdir bin dev etc lib tmp
# mkdir usr/lib -p
# mkdir var/run/apache2 -p
# mkdir var/run/mysqld -p
# chown mysql:mysql var/run/mysqld
# chmod a=rwxt tmp

# mkdir siteroot/site1/htdocs -p
# mkdir siteroot/site1/logs -p
# mkdir siteroot/site2/htdocs -p
# mkdir siteroot/site2/logs -p

# mknod -m 666 dev/null c 1 3
# mknod -m 666 dev/zero c 1 5
# mknod -m 644 dev/random c 1 8
# mknod -m 644 dev/urandom c 1 9

For amd64 installations only

# ln -s lib lib64
# ln -s usr/lib usr/lib64

We need to copy over some essential libs for dns lookups.

# cp /lib/libnss_compat.so.2 /wwwjail/lib
# cp /lib/libnss_dns.so.2 /wwwjail/lib
# cp /lib/libnss_files.so.2 /wwwjail/lib
# cp /lib/libresolv.so.2 /wwwjail/lib
# cp /lib/ld-linux.so.2 /wwwjail/lib
# cp /lib/libc.so.6 /wwwjail/lib
# cp /lib/libnsl.so.1 /wwwjail/lib
# cp /etc/resolv.conf /etc/host.conf /etc/hosts /wwwjail/etc

Finally copy over the users, groups, and a few other misc files. You may want to modify the passwd and gorup files to only include users and groups you need. Please note we have omitted the shadow file. If you need users to log into the jail then you may copy the file over and manually edit it!

# cp /etc/passwd /wwwjail/etc/
# cp /etc/group /wwwjail/etc/
# cp /etc/ld.so.* /wwwjail/etc/
# cp /etc/mime.types /wwwjail/etc/

Building the chroot with jail

Please skip this section if you plan on building the chroot manually.

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   ] app-misc/jail-1.9-r2  0 kB

You might want to create a user that will be jailed inside of apache's chroot for users that maybe maintaining the code. In this example I will add a user and group for wwwdev. You could add additional accounts with that group if needed.

# groupadd wwwdev
# useradd -g users wwwdev -d /wwwjail -s /usr/bin/jail wwwdev 

Now that we have a user we need to make the jail environment.

# mkdir /wwwjail
# mkjailenv /wwwjail
Warning: mkjailenv will copy over your /etc/passwd and /etc/shadow files. It is highly recommended that you modify these files to have only uid's needed by the jail. (REMOVE ROOT)

At this point you should have a few libs and files in your jail all populated thansk to mkjailenv. If you would like to add additional software like bash and curl you need to add them to the jail.

# addjailsw /wwwjail -P bash "--version"
# addjailsw /wwwjail -P curl
Note: For some reason, the addjailsw tool does not fetch the ld-linux.so.2, which leads to the error "execve(): File or Directory doesn't exist", so we copy it manually.

Gentoo x86

# cp /lib/ld-linux.so.2 /wwwjail/lib/

Gentoo x86_64

# mkdir /wwwjail/lib64/
# cp /lib64/ld-linux-x86-64.so.2 /wwwjail/lib64/

If you would like to just move over the standard programs like cp, cat, grep, mkdir, ls, etc. just issue the following command.

# addjailsw /wwwjail -D

Adding apache to the chroot

Now comes the time to move apache into the jail. You can get away with not copying some of these files however the graceful restart (aka reload) will not work and cause apache to crash. I would recommend moving these files by hand as addjailsw has not been tested!

First move the apache configuration files. We will symlink them back to their original locations to prevent confusion and to allow programs/updates to get to the correct location. This will allow our apache conf files to refer to a path that is valid both before and after jailing. The additional symlink wwwjail -> . will allow are document roots for our hosts to have a consistent path for before apache is jailed (at cold start) and after the chroot has taken place (graceful restart).

# mv /etc/apache2 /wwwjail/etc/
# ln -s /wwwjail/etc/apache2 /etc/apache2
# mkdir /wwwjail/etc/conf.d -p
# mv /etc/conf.d/apache2 /wwwjail/etc/conf.d/apache2
# ln -s /wwwjail/etc/conf.d/apache2 /etc/conf.d/apache2
# mv /var/log/apache2 /wwwjail/var/log/apache2
# ln -s /wwwjail/var/log/apache2 /var/log/apache2

Now we need to move the apache binaries to the jail.

# mv /usr/lib/apache2 /wwwjail/usr/lib/
# ln -s /wwwjail/usr/lib/apache2 /usr/lib/apache2

Now we get to the hard part. The apache modules require libs that are not currently in our chrooted environment. So you need to goto /wwwjail/usr/lib/apache2/modules and basically run ldd on each of the module files and copy the depends that are not currently in the jail... This is extremely time consuming so I have written a script to do this for you.

File: jail_fixlibs.sh
#!/bin/sh

for file in `find /wwwjail/usr/lib/apache2/modules -name "*.so" -exec echo {} \;` ; do
        ldd $file | grep "/" | awk '{print $3}' | xargs -r -n 1 echo -e -e | while read aLine ; do
                echo -e "Copying File: $aLine"
                cp $aLine /wwwjail$aLine
        done
done

Configuring Apache

First thing to do is the stop apache to prevent some rouge pid and lock files.

# /etc/init.d/apache2 stop


The apache pid and lock file will need to be moved to the chroot so that apache status and graceful restart will function correctly. Without the pid file the daemon script can not find the running apache process.

File: /etc/apache2/modules.d/00_mpm.conf
# PidFile: The file in which the server should record its process
# identification number when it starts.
#
# Note that this is the default PidFile for most MPMs.
PidFile /wwwjail/var/run/apache2.pid

# The accept serialization lock file MUST BE STORED ON A LOCAL DISK.
LockFile /wwwjail/var/run/apache2.lock


Now we need to tell mod_security where our jail is located so add a line at the end of /etc/apache2/modules.d/99_mod_security.conf:

File: /etc/apache2/modules.d/99_mod_security.conf
        # use Core Rule Set by default:
        Include /etc/apache2/modules.d/mod_security/*.conf

        SecChrootDir /wwwjail
</IfDefine>


You need to tell apache to load mod_security by specifying the "-D SECURITY" option.

File: /etc/conf.d/apache2
APACHE2_OPTS="-D SECURITY -D DEFAULT_VHOST -D INFO -D LANGUAGE -D SSL -D SSL_DEFAULT_VHOST -D SUEXEC -D PHP5"


Due to heavy modification I have included a drop-in replacement to the standard Gentoo init.d script.

File: /etc/init.d/apache2
#!/sbin/runscript
# Copyright 1999-2007 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

opts="configtest fullstatus graceful gracefulstop modules reload"

depend() {
        need net
        use mysql dns logger netmount postgresql
        after sshd
}

configtest() {
        ebegin "Checking Apache Configuration"
        checkconfig
        eend $?
}

checkconfig() {
        SERVERROOT="${SERVERROOT:-/usr/lib/apache2}"
        if [ ! -d ${SERVERROOT} ]; then
                eerror "SERVERROOT does not exist: ${SERVERROOT}"
                return 1
        fi

        CONFIGFILE="${CONFIGFILE:-/etc/apache2/httpd.conf}"
        [ "${CONFIGFILE#/}" = "${CONFIGFILE}" ] && CONFIGFILE="${SERVERROOT}/${CONFIGFILE}"
        if [ ! -r "${CONFIGFILE}" ]; then
                eerror "Unable to read configuration file: ${CONFIGFILE}"
                return 1
        fi

        APACHE2_OPTS="${APACHE2_OPTS} -d ${SERVERROOT}"
        APACHE2_OPTS="${APACHE2_OPTS} -f ${CONFIGFILE}"
        [ -n "${STARTUPERRORLOG}" ] && APACHE2_OPTS="${APACHE2_OPTS} -E ${STARTUPERRORLOG}"

        APACHE2="/usr/sbin/apache2"

        ${APACHE2} ${APACHE2_OPTS} -t 1>/dev/null 2>&1
        ret=$?
        if [ $ret -ne 0 ]; then
                eerror "Apache2 has detected a syntax error in your configuration files:"
                ${APACHE2} ${APACHE2_OPTS} -t
        fi

        return $ret
}

start() {
        checkconfig || return 1
        ebegin "Starting apache2"
        [ -f /var/log/apache2/ssl_scache ] && rm /var/log/apache2/ssl_scache

        #Original Code: start-stop-daemon --start --exec ${APACHE2} -- ${APACHE2_OPTS} -k start
        ${APACHE2} ${APACHE2_OPTS} -k start
        eend $?
}

stop() {
        checkconfig || return 1
        ebegin "Stopping apache2"

        #Original Code: start-stop-daemon --stop --retry -TERM/5/-KILL/5 --exec ${APACHE2} --pidfile /var/run/apache2.pid
        ${APACHE2} ${APACHE2_OPTS} -k stop
        eend $?
}

reload() {
        RELOAD_TYPE="${RELOAD_TYPE:-graceful}"

        checkconfig || return 1
        if [ "${RELOAD_TYPE}" = "restart" ]; then
                svc_restart
                
		# Original Code
                #RELOAD_SIGNAL="HUP"
                #start-stop-daemon --stop --oknodo --signal HUP --exec ${APACHE2} --pidfile /var/run/apache2.pid
        elif [ "${RELOAD_TYPE}" = "graceful" ]; then
                ebegin "Reloading apache2"
                ${APACHE2} ${APACHE2_OPTS} -k graceful
                eend $?
		
		# Original Code
                #RELOAD_SIGNAL="USR1"
                #start-stop-daemon --stop --oknodo --signal USR1 --exec ${APACHE2} --pidfile /var/run/apache2.pid
        else
                eerror "${RELOAD_TYPE} is not a valid RELOAD_TYPE. Please edit /etc/conf.d/apache2"
        fi
}

modules() {
        checkconfig || return 1

        ${APACHE2} ${APACHE2_OPTS} -M 2>&1
}

fullstatus() {
	# This is kind of a usless feature...
	checkconfig || return 1

        if pgrep -f "${APACHE2} .*" >/dev/null ; then
                einfo "http status:\tstarted"
        else
                eerror "http status:\tstopped"
        fi
}


The last thing to do is to modify your apache hosts to point to the jail. You will need to move all your files to the appropriate folders in the jail that we setup (/wwwjail/siteroot/site1/htdocs). You will also notice that we created some log folders. You can setup your hosts to use these folders if you would like. You need to make sure you change the paths to doc roots and anything else you may have to point to the chroot.

        DocumentRoot "/wwwjail/siteroot/site1/htdocs"

        # This should be changed to whatever you set DocumentRoot to.
        <Directory "/wwwjail/siteroot/site1/htdocs">
                Options Indexes FollowSymLinks

                # AllowOverride controls what directives may be placed in .htaccess files.
                # It can be "All", "None", or any combination of the keywords:
                #   Options FileInfo AuthConfig Limit
                AllowOverride All

                # Controls who can get stuff from this server.
                Order allow,deny
                Allow from all
        </Directory>


Now start apache back up and hope for the best.

/etc/init.d/apache2 start

In the event of an issue with apache starting you can manually kill the process using the following commands.

# pkill -9 apache2
# /etc/init.d/apache2 zap

Configuring MySql

We need to update the mysql config to put some important files in the jail. The reason we have to include some files in the jail (more specifically the sock and pid files) is because of the way linux will connect to mysql if connecting to the localhost. If the system knows your connecting to the localhost it will use domain socket instead of a tcp socket because it is much faster.

First you need to stop mysql to make sure the old pid and sock files are cleaned up.

# /etc/init.d/mysql stop

Then edit the config file to point mysql pid and sock files to the jail.

File: /etc/mysql/my.cnf
[client]
socket            = /wwwjail/var/run/mysqld/mysqld.sock

[mysqld]
socket            = /wwwjail/var/run/mysqld/mysqld.sock
pid-file          = /wwwjail/var/run/mysqld/mysqld.pid

Now that the config file is pointing to the jail start mysql again.

# /etc/init.d/mysql start

You should see the pid and socket files located in the chroot.

# ls -la /wwwjail/var/run/mysqld/
total 620
drwxr-xr-x 2 mysql mysql   4096 Oct  4 15:23 .
drwxr-xr-x 4 root  root    4096 Oct  3 17:08 ..
-rw-rw---- 1 mysql mysql      5 Oct  4 15:23 mysqld.pid
srwxrwxrwx 1 mysql mysql      0 Oct  4 15:23 mysqld.sock

Other settings

Here are some additional items that maybe required by your chroot environment. It is highly recommended that you add a shell and some type of mta.

Adding a shell

Some features, such as mail, need a shell inside the jail.

Adding Bash

Please skip this section if you used jail to build the chroot.
If you prefer sash, as your chroot shell, jump to next point.

To have bash, do:

# mkdir /chroot/apache/bin
# cp /bin/bash /chroot/apache/bin
# ln -s bash /chroot/apache/bin/sh
# ldd /chroot/apache/bin/bash
        libncurses.so.5 => /lib/libncurses.so.5 (0x00002b26cf97d000)
        libdl.so.2 => /lib/libdl.so.2 (0x00002b26cfada000)
        libc.so.6 => /lib/libc.so.6 (0x00002b26cfbde000)
        /lib64/ld-linux-x86-64.so.2 (0x00002b26cf865000)
# cp /lib64/ld-linux-x86-64.so.2 /chroot/apache/lib/
# cp /lib/libc.so.6 /chroot/apache/lib/
# cp /lib/libdl.so.2 /chroot/apache/lib
# cp /lib/libncurses.so.5 /chroot/apache/lib
# chroot /chroot/apache
bash-3.1# # enjoy!
bash-3.1# exit
exit

Adding csh

Instead of building a complete bash environment, may you choose sash, a light, statically linked shell executable with many built-in commands. You don't need to add any additional libraries, and security should be enhanced. With sash, sending mail works fine but please test all other features carefully.

# USE="-* readline" FEATURES="nodoc noman noinfo" ROOT=/wwwjail INSTALL_MASK="*.h" emerge -av sash
# ln -sf /wwwjail/bin/sash /wwwjail/bin/sh

For testing:

# chroot /wwwjail /bin/sh
Stand-alone shell (version 3.7)
$ -ls
.
..
bin
chroot
dev
etc
lib
lib64
modules.d
pippo
root
tmp
usr
var
www
$ # enjoy
$ quit

Adding an MTA

You have a few choices for an MTA. As of writing this document I have only tested ssmtp.

Sending mail with ssmtp

Apache often needs a way to send mail. Have you just a working ssmtp on your system?

If no, that may be a good example for a gmail setting.

Anyway, let assume this command is correctly working in your sistem:

# echo ""|ssmtp -s foo pluto@gmail.com

Compile ssmtp in the chroot:

# USE="-* ssl readline" FEATURES="nodoc noman noinfo" ROOT=/wwwjail INSTALL_MASK="*.h" emerge -av ssmtp

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    ] net-mail/mailbase-1 to /chroot/apache/ USE="-pam"
[ebuild  N    ] dev-libs/openssl-0.9.7j to /chroot/apache/ USE="-bindist -emacs -test -zlib"
[ebuild  N    ] mail-mta/ssmtp-2.61 to /chroot/apache/ USE="ssl -ipv6 -mailwrapper -md5sum"
[ebuild  N    ] app-misc/ca-certificates-20050804 to /chroot/apache/

Would you like to merge these packages? [Yes/No]


Add some libraries and configuration files:

# ldd /chroot/apache/usr/sbin/ssmtp
  libnsl.so.1 => /lib/libnsl.so.1 (0x00002b47ef97b000)
  libssl.so.0.9.7 => /usr/lib/libssl.so.0.9.7 (0x00002b47efa91000)
  libc.so.6 => /lib/libc.so.6 (0x00002b47efbcb000)
  libcrypto.so.0.9.7 => /usr/lib/libcrypto.so.0.9.7 (0x00002b47efdf3000)
  libdl.so.2 => /lib/libdl.so.2 (0x00002b47f003c000)
  /lib64/ld-linux-x86-64.so.2 (0x00002b47ef863000)
# cp /lib/libnsl.so.1 /chroot/apache/lib
# cp /lib/libssl.so.0.9.7 /chroot/apache/lib
# cp /lib/libcrypto.so.0.9.7 /chroot/apache/lib
# cp /lib/libnss_compat.so.2 /chroot/apache/lib
# cp /lib/libnss_dns.so.2 /chroot/apache/lib
# cp /lib/libnss_files.so.2 /chroot/apache/lib
# cp /lib/libresolv.so.2 /chroot/apache/lib
# cp /etc/resolv.conf /etc/host.conf /etc/hosts /chroot/apache/etc
# cp /etc/ssmtp/ssmtp.conf /etc/ssmtp
# echo root:x:0:0:root:/root:/bin/bash>pippo/etc/passwd

Ssmtp needs a passwd entry for ssmtp and apache. Double Check?

And now test if everything looks ok:

# chroot /chroot/apache
bash-3.1#  echo ""|ssmtp -s pippo ziapannocchia@gmail.com


Finally: ssmtp needs to exchange a random number with remote email-server. Please make sure the following devices were created.

# mknod /chroot/apache/dev/null c 1 3
# mknod /chroot/apache/dev/urandom c 1 9

Test your mail service and enjoy.

Sending mail with mini-qmail

Untested, may only work with a qmail server.

Sending mail with mini-sendmail

Untested, no ebuild available! http://www.acme.com/software/mini_sendmail/

mod_fcgid or mod_fastcgi

For cgi support you will need to point the cgisock to the chroot. I do not need cgi support so if you use it please complete this section.

File: Unknown
# Processor user group /path/to/chroot
Scriptsock            /wwwjail/var/run/apache2/cgisock

mod_security

We have only configured mod_security to provide a chroot environment. Mod security can be used to secure apache further by providing protection against a wide range of attacks including sql injection.

To learn more about setting up a really safe mod_security configuration, read original documentation.

Have also a look to these examples:

mediawiki

Testing mediawiki, logs may give this error:

Allowed memory size of 8388608 bytes exhausted (tried to allocate 203 bytes)

May you solve everything copying /etc/php in the jail:

# cp -a /etc/php /chroot/apache/etc

mod_python

To use mod_python, you need a complete python environment inside the chroot.

I don't know correlated security holes. Go ahead at your own risk.

Anyway, may you setup mod_python in such a way:

# ROOT="/chroot/apache" emerge --nodeps python mod_python
File: /etc/apache2/modules.d/16_mod_python.conf
...
</IfDefine>
AddHandler mod_python .psp
PythonHandler mod_python.psp
...

Conclusions

Now you should have a working environment to protect your system from your websites. Please note that this configuration will not protect your virtual hosts from each other. If you would like to protect your vhosts from each other you should look into reverse proxy setup. I will be creating an article for that next.

Trouble Shooting

If you run into issues where your jailed site stops working after a period of time it is most likely due to issues with missing files in the jail. To confirm this restart apache and then do a reload (graceful restart). If you notice that the site dies, check the logs files. More thank likely it will complain about something missing such as a shared object, config file, or log.

I have created a script that will go through and try and fix some of the missing file problems.

File: jail_update.sh
#!/bin/bash

##cp the /etc/ld.so.conf so we can find our libs
echo -e "Updating /etc/ld.so.conf"
cp -vf /etc/ld.so.* /wwwjail/etc/

##Copy over the user and group files
echo -e "Updating jail users and groups"
cp -vf /etc/passwd /wwwjail/etc/
cp -vf /etc/group /wwwjail/etc/

##Copy over system mime types
echo -e "Updating system mime types"
cp -vf /etc/mime.types /wwwjail/etc/

## Install Sendmail application
echo -e "Checking for ssmtp updates..."
USE="-* ssl" FEATURES="nodoc noman noinfo" ROOT=/wwwjail INSTALL_MASK="*.h" emerge -v --update ssmtp

echo -e -e "\nUpdating ssmtp libs..."
ldd /wwwjail/usr/sbin/ssmtp  | grep "/" | awk '{print $3}' | xargs -r -n 1 echo -e -e | while read aLine ; do
        echo -e -e "Copying File: $aLine"
        cp $aLine /wwwjail$aLine
done
echo -e "Updating ssmtp conf..."
cp /etc/ssmtp/ssmtp.conf /wwwjail/etc/ssmtp


##Copy files for dns
echo -e "\nUpdating dns libs..."

echo -e "Copying File: /lib/libnss_compat.so.2"
cp /lib/libnss_compat.so.2 /wwwjail/lib

echo -e "Copying File: /lib/libnss_dns.so.2"
cp /lib/libnss_dns.so.2 /wwwjail/lib

echo -e "Copying File: /lib/libnss_files.so.2"
cp /lib/libnss_files.so.2 /wwwjail/lib

echo -e "Copying File: /lib/libresolv.so.2"
cp /lib/libresolv.so.2 /wwwjail/lib

echo -e "Copying File: /lib/ld-linux.so.2"
cp /lib/ld-linux.so.2 /wwwjail/lib

echo -e "Updating dns conf..."
cp /etc/resolv.conf /etc/host.conf /etc/hosts /wwwjail/etc


##Installing bash
echo -e "\nUpdating bash libs..."
echo -e "Copying File: /bin/bash"
cp /bin/bash /wwwjail/bin

echo -e "Linking File: /bin/sh"
ln -s bash /wwwjail/bin/sh
ldd /wwwjail/bin/bash  | grep "/" | awk '{print $3}' | xargs -r -n 1 echo -e -e | while read aLine ; do
        echo -e "Copying File: $aLine"
        cp $aLine /wwwjail$aLine
done

## Fix any missing shared objects.
for file in `find /wwwjail/usr/lib/ -name "*.so*" -exec echo {} \;` ; do
        ldd $file | grep "/" | awk '{print $3}' | xargs -r -n 1 echo -e -e | while read aLine ; do
                echo -e "Copying File: $aLine"
                cp $aLine /wwwjail$aLine
        done
done

for file in `find /wwwjail/lib/ -name "*.so*" -exec echo {} \;` ; do
        ldd $file | grep "/" | awk '{print $3}' | xargs -r -n 1 echo -e -e | while read aLine ; do
                echo -e "Copying File: $aLine"
                cp $aLine /wwwjail$aLine
        done
done

References

Retrieved from "http://www.gentoo-wiki.info/Apache_chroot:_the_mod_security_way"

Last modified: Thu, 20 Mar 2008 00:22:00 +0000 Hits: 17,115