06- Apache 2.2.15 on CentOS 6.2 x64

Starting Apache

Apache is installed on the virtual machines provided in class. To verify its status, you can run either

[root@dollysisters ~]# service httpd status
httpd (pid  2357) is running...
[root@dollysisters ~]# /etc/init.d/httpd status
httpd (pid  2357) is running...

It can be started, stopped, or restarted by using the appropriate verb.

To ensure it always starts, navigate System → Administration → Server Settings → Services and enable httpd. This can be done manually via chkconfig.

In either case, be sure to open the firewall: navigate System → Administration → Firewall.

Version

To find the version of apache, run

# httpd -v

This document describes the 2.2.x series for Apache, from CentOS 5.5. Other versions of Apache include the 1.x series and the 2.0.x series, which have slightly different internal structures. Version 2.4 was released earlier this year, and will also not be covered.

Documentation

Apache comes with its own documentation. Suppose that your web server is located at “dollysisters.cosc.tu”. Then, at least initially, the apache manual is accessible at http://dollysisters.cosc.tu

Verify this, by loading the manual- both from a browser on the server itself, and from a second browser on a different host.

Apache Structure

Apache is structured around a series of modules, which can either be compiled into the program or added dynamically. To see the compiled modules, run

[root@dollysisters ~]# httpd -l

How many complied in modules are present in your system?

To see the dynamically loaded modules, we will need to look at the configuration file- later!

Directories and files

These are the CentOS defaults; other systems use variations on this theme.

/etc/httpd/
    conf/
        httpd.conf      Main configuration file
        magic
    conf.d/             Subdirectories for specific tasks
        manual.conf     Should the apache manual be served?
        mod_dnssd.conf
        nss.conf
        php.conf        Set up PHP for apache; not the same as php.ini
        README
        ssl.conf        Configure SSL
        welcome.conf
    logs ->              link to /var/log/httpd/
    modules ->           link to /usr/lib64/httpd/modules/
    run ->               link to var/run/
/var/log/httpd/
    access_log
    error_log
    ssl_access_log
    ssl_error_log
    ssl_request_log
/var/www/
    cgi-bin/            directory for CGI script
    error/              HTTP error messages in multiple languages
    html/               Root directory for web
    icons/              Collection of public doman icons
    manual/             http://localhost/manual

Because of the size and complexity of this system, and because we have only a limited time to devote to this topic, we will only cover some of the functionality available in the system. You are encouraged to explore.

Tour of the configuration files

/etc/httpd/conf/httpd.conf

Line 57 sets the root directory for the server (not the web site(s))

ServerRoot "/etc/httpd"

Read http://localhost/manual/mod/core.html#serverroot

Line 136 sets the IP addressed and port(s) that apache is litening for

Listen 80

The default is to listen to all IP adresses of the host, on port 80. See later in this document for details on how to set your Linux system up to use multiple interfaces.

Read http://localhost/manual/mod/mpm_common.html#listen for details on the directive.

Lines 150++ load various dynamic modules.
<pre
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule auth_digest_module modules/mod_auth_digest.so

How many dynamic modules does Apache load? Does this explain why they were not listed earlier in our discussion of compiled modules?

To find out more about these modules, read http://localhost/manual/mod.

Line 221 specifies additional configuration files to read

Include conf.d/*.conf

This ensures that the configuration files in this subdirectory are also loaded by Apache. See http://localhost/manual/mod/core.html#include for the syntax.

Lines 242-243 set the user and group for the server

User apache
Group apache

See http://localhost/manual/mod/mpm_common.html#user

Line 276 sets the name of the server; it should be configured to match your hostname. In this example, the FQDN of the server is dollysistsers.cosc.tu, so we set

ServerName dollysisters.cosc.tu
Making changes to Apache configuration files

Once changes are made to any of the configuration files, we need to tell Apache to reread the configuration files. Clearly this can be done with the restart verb in either

[root@dollysisters html]# /etc/init.d/httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
[root@dollysisters html]# service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
[root@dollysisters html]# 

This does have a downside though- this would disrupt any current connections to the server; thus the verb reload is preferred, using either

[root@dollysisters html]# /etc/init.d/httpd reload
Reloading httpd: 
[root@dollysisters html]# service httpd reload
Reloading httpd: 
[root@dollysisters html]# 

This will reload the configuration files, without stopping the server.

Now I know that no one in my class could ever make a syntax error in a configuration file, but it is possible that you may see it happen to someone else. If there is a syntax error in the configuration files, the reload will fail and the server will continue with the previous configuration file.

You can validate your configuration file before reloading or restarting. Suppose that we deliberately introduce a simple error into our configuration file, replacing the ServerName directive with

ServerName dollysist
ers.cosc.tu

Here we have "accidentally" split the hostname on to two lines. Clearly the second line is a syntax error. Attempting to reload the configuration throws an error:

Reloading httpd: not reloading due to configuration syntax error
                                                           [FAILED]

but no information is provided as to the source of the problem. Here we run

[root@dollysisters html]# apachectl configtest
Syntax error on line 277 of /etc/httpd/conf/httpd.conf:
Invalid command 'ers.cosc.tu', perhaps misspelled or defined by a module not 
included in the server configuration

to locate the error.

Returning to our tour of /etc/httpd/conf/httpd.conf, line 294 sets the root directory for the web site (not the server)

DocumentRoot "/var/www/html"

See http://localhost/manual/mod/core.html#documentroot

Apache sets policies on a directory-by-directory basis; moreover these directory directives can overlap. Read the documentation at http://localhost/manual/mod/core.html#directory

What is the default policy of Apache towards a directory (Lines 302-305)?

Note that the syntax for the options for the <Directory> directive are not described at http://localhost/manual/mod/core.html#directory Instead, look at http://localhost/manual/mod/core.html#options to see the meaning of the options.

Read the documentation at http://localhost/manual/sections.html. What happens if two different <Directory> directives apply to a directory? Which takes precedence?

Lines 317-346 form another <Directory> directive. What is the directory to which they refer? Create a valid html file in that directory. Verify that it is served. Create a symlink to /etc/passwd and put it in the directory. What happens when that file is requested? Create a hard link to /etc/passwd and put it in the directory. What happens when that file is requested? Repeat the previous with /etc/shadow. Explain your observations.

What are the access control options for Document Root (lines 343-344)? See http://localhost/manual/mod/mod_authz_host.html for details of the syntax.

If the Allow from all line is removed, what is the default access policy for the server? Verify this.

Lines 361-375 by default turn off the use of directories for individual users. See http://localhost/manual/mod/mod_userdir.html and see also http://localhost/manual/howto/public_html.html

Enable user directories, but only for one user. To do so for the user vimes, modify these lines to include:

UserDir disable
UserDir enabled vimes

Why is this approach preferred over enabling user directories for all users?

Set the directory that is to be shared to be public_html by uncommenting the line

UserDir public_html

Be sure to test then reload your configuration file.

If a request is made of the server for the file http://server/~user/dir/page.html where in the file system should that file be located? Create this directory and this file. Test to see if the server serves your file. It didn’t? Huh. What are the permissions of the directory /home/user? What permissions are necessary so that another user (say the user apache from group apache) can traverse this directory? How about we make that change! What are the permissions for the directory and the file for your web page? Test again!

The joy of SELinux

It still doesn’t work even though the permissions are right? Welcome again to SELinux. There are people who think that this is a great tool for securing a Linux system. Other people. It turns out that SELinux is preventing apache from reading the file you have set up because it is in a different security context. This would not be a huge hurdle, provided SELinux would explain what was being blocked and why. There are tools to try and figure out what SELinux is doing and why certain actions are being blocked, and that program is installed by default. Oh silly me, no it isn’t- that would take the fun out it.

Let’s solve our SELinux issue with the file. Run the command

[root@dollysisters selinux]# echo 0 >/selinux/enforce 

This will persist until the next reboot. To make the change permanent, open /etc/selinux/config and set

SELINUX=permissive

Now you may feel the urge to set that all the way to disabled. If you do though, you will have difficulty enabling SELinux should you later change your mind.

Returning to our original question. Suppose that you let individual users set up their own web pages on a server in this fashion. How can this information be used by an attacker to determine the users of a computer?

Lines 381-392 are the <Directory> directive for the use of directories by individual users. Note that, they are commented out in the default configuration, meaning they fall back to the default Directory policy (lines 302-305).

To understand the syntax for the <Limit> directive, see http://localhost/manual/mod/core.html#limit, while to understand the syntax for the <LimitExcept> directive, see http://localhost/manual/mod/core.html#limitexcept

What HTTP methods are allowed for the public user directories? What HTTP methods are not allowed for the public user directories?

Line 402 determines what file will be served if the user asks for a directory.

Line 409 sets the default name (.htaccess) for directory-by-directory additional configuration files. For details on how this file is used, see http://localhost/manual/mod/core.html#accessfilename

Will .htaccess files be read for directories within DocumentRoot (/var/www/html)? Hint: What happens on line 338? What about the default? (Hint: Line 304.)

Lines 415-419 prevent any files that start with the characters .ht from being served See http://localhost/manual/mod/core.html#files for documentation on the Files directive. If you want to, know more about regular expressions, go online- just remember that (1) Apache uses PCRE and (2) "Some people, when confronted with a problem, think ‘I know, I’ll use regular expressions.’ Now they have two problems."

Line 484 sets the location of the error log.

ErrorLog logs/error_log

Note that this is relative to the root directory for Apache configuration (/etc/httpd) and should now explain why that directory has a symlink from logs to /var/log/httpd.

See http://localhost/manual/mod/core.html#errorlog for details; notice also that the log system can be set to use syslog.

Line 491 tells the server to log only at the level of warn or higher

LogLevel warn

See http://localhost/manual/mod/core.html#loglevel for the details.

Lines 497-526 set up the access logs for the system; these will be discussed in some detail below.

Line 536 tells Apache to include information about the current server version when returning an error message. See http://localhost/manual/mod/core.html#serversignature

Lines 551-558 set up an alias for the icons directory. View http://localhost/icons/.
(The trailing / is necessary.) Note the list of icons- even though there is no icons directory in document root- the alias command created it.

If an alias points outside of documentroot, permissions need to manually set on the directory. Aliases are quite powerful- see http://localhost/manual/mod/mod_alias.html

Line 576-587 sets up an alias for CG scripts; this alias is similar to the alias described above. To practice with a CGI script, create the file test.pl with the following content:

#!/usr/bin/perl
print "Content-type: text/html\n\n";
print "This is a cgi script\n";

Place the file in the CGI directory, and make it world-executable.

The first line tells the system how to execute the file. The second sets up the headers (and it needs BOTH newlines). The rest- well you can have it do whatever you want at that point. Note that this is code that is being executed by the server, on the server. Then http://localhost/cgi-bin/test.pl will execute the script and return the result.

It is possible to set up per-user CGI script directories if individual users have their own web directories. See http://localhost/manual/howto/cgi.html for more on CGI scripting. See http://localhost/manual/mod/mod_alias.html#scriptalias for more on the syntax for ScriptAlias

There is more to the file, but it is of lesser importance than the above, and we do nto have time for the discussions.

/etc/httpd/conf.d/ssl.conf
This is used to configure SSL on the server. We will discuss this in detail later when we cover SSL.

Note that this file is automatically included in the Apache configuration at line 221 of /etc/httpd/httpd.conf, where the directive

Include conf.d/*.conf

exists. Note that the structure of this include statement only includes files that end in “.conf”. Because many text editors automatically create backups of files, if you are not careful with this directive you can end up including both the original and the backup file, duplicating some statements and causing no end of hassle.

/etc/httpd.conf/manual.conf
The file manual.conf is an alias and a directory directive to allow Apache to display the manual. This is an allow from all- so the user does not have to be on localhost.

You can disable the manual by editing this configuration file.

/etc/httpd.conf/php.conf
Configures how Apache interacts with PHP. This is not the same as the configuration for PHP itself; this is usually /etc/php.ini

This file also tells Apache that, if a directory in requested and the file index.php exists in that directory, then to return that file (Line 22).

The file welcome.conf will display the Apache welcome if no default index page exists.

Hardening

Apache should have a separate user:group to run the server. (It does- apache:apache by default in CentOS). It does need to start as root so that it can bind to port 80. The choice is made in httpd.conf, lines 242-243. What would happen if this file is writable by other than root?

Permissions on the binaries need to be set correctly. What are the default permissions on the executable for the apache web server? [Indeed- what is the name of the executable?]

Permissions on the logs should be set so that only root can read them. What are the permissions on the directory /var/log/httpd? Why? These will store user supplied data- if they can get a process to read the data, it might be “trusted” as it comes from the local file system.

There is a separate file clobbering issuse described in http://localhost/manual/misc/security_tips.html#serverroot

Logging

NOTE- This is for non-SSL logging. Logs for SSL connections will be described later.

There are two kinds of logging- Request logging and Error logging.

Request logging
One must first create a log format using a LogFormat directive. See either http://localhost/manual/mod/mod_log_config.html#logformat or http://localhost/manual/mod/mod_log_config.html#formats for more details.

Four common log formats types are defined:

  • combined
  • common
  • referer
  • agent

By default, Apache uses the combined format. [See line 526.]

Logging itself can be accomplished with one of two commands:

CustomLog

(http://localhost/manual/mod/mod_log_config.html#customlog) or

TransferLog

(http://localhost/manual/mod/mod_log_config.html#transferlog). The second is rarely used.

Error Logging
Error logs are specified with the ErrorLog directive. [See line 484]. Read http://localhost/manual/mod/core.html#errorlog for details.

Error logs are written according to a priority specified by the LogLevel directive. Error levels are:

  • emerg Emergencies – system is unusable.
  • alert Action must be taken immediately.
  • crit Critical Conditions.
  • error Error conditions.
  • warn Warning conditions.
  • notice Normal but significant condition.
  • info Informational.
  • debug Debug level messages.

Error logs can be sent to

  • A file- e.g.
    ErrorLog /var/log/httpd/error_log
  • An application that will handle the data- e.g.
    ErrorLog "|/usr/local/bin/httpd_errors"

    Note that the programs so started are started by the web server, and may have root privileges.

  • To syslog, specifying a facility and priority- e.g.
    ErrorLog syslog:user

Try some of these different options.

Virtual Hosts

Virtual hosts are a way that a single web server can run multiple web sites.

  • You can have one host with multiple DNS names serve different websites depending on the hostname in the request
  • You can have one host with multiple IP addresses serve different websites depending on the IP address in the request.
  • You can have one host serve different websites depending on the port in the request.

These different web sites can have different

  • Server admin
  • Document root
  • Server Name

Apache uses virtual hosts as a key mechanism for SSL protected web sites.

Log files
See http://localhost/manual/mod/core.html#virtualhost, http://localhost/manual/vhosts/, and http://localhost/manual/vhosts/examples.html for additional documentation.

What happens is the same directory is accessible to two or more virtual hosts? They both can access it! This can cause security issues, as we will shortly see.

SSL

The configuration for the SSL secured component of a website is contained in the file /etc/httpd/conf.d/ssl.conf

Line 12 loads the module needed for SSL

By default, SSL listens on port 443; this is enabled on line 18.

Lines 20-68 set options that are global to all SSL protected web sites on that server, including properties of the random number generators and the cache.

To configure a portion of the website to use SSL, Apache uses a <VirtualHost> directive, in lines 74-221. Note first that the directive

<VirtualHost _default_:443>

matches any IP address not listed in another <VirtualHost> directive, on port 443. [c.f. http://localhost/manual/mod/core.html#virtualhost%5D.

Virtual hosts can specify their own document root separately from the main host; thus we can specify a different document root for the SSL portion of the web site. In the default configuration file, line 77, the DocumentRoot directive is commented out, and hence anything in the DocumentRoot for the main web site can also be served via SSL.

Recall, if a directory is accessible via two <VirtualHost> directives, then both directories will serve that document. Verify this behavior. Earlier you created a file for your web server to serve. Now access it via SSL (i.e. https://yoursite/yourfile). Ignore any SSL warnings for the moment, and verify that you get the same page.

Line 78 sets the server name for this virtual host, or it would it it were not commented out. Remember that one Apache server can use virtual hosts with different host names. See http://localhost/manual/mod/core.html#servername for more details.

Line 82 sets the location of the error log, line 83 sets the location of the transfer log and line 84 sets the log level for this virtual host.

Line 88 enables SSL for access to this virtual host.

Remember- the virtual host directive here was <VirtualHost _default_:443>, so anything that comes in via port 443 to any IP adress not bound to another virtual host will use SSL. However- this does not mean that any access to a directory inside this virtual host will automatically require SSL. If a file is in the document root for more than one virtual host, then either host can serve that file! We have seen this already!

Don’t confuse the fact that the virtual host uses SSL with the notion that the files necessarily require SSL- that depends on the totality of your Apache configuration.

See http://localhost/manual/mod/mod_ssl.html#sslengine for the syntax

The directive SSLProtocol on line 93 specifies the versions of SSL that will be accepted. See http://localhost/manual/mod/mod_ssl.html#sslprotocol for the syntax. Note that SSLv2 is known to be flawed; it is disabled by default in modern browsers (e.g. IE7+, FF2+)

The precise Cipher is set by the SSLCipherSuite directive in line 98.

The private and public key locations are set in lines 105 & 112.

Read the remainder of the configuration file; note the additional log statement in lines 218-219.

To illustrate, let us set up an SSL protected directory on our web server that is only accessible via SSL.

Create the directory /var/www/html/ssl. Be sure that permissions are set so that it can be read by user apache of group apache.

Create the file /var/www/html/ssl/index.html.

Verify that right now this file can be accessed via both ssl (https) and non-ssl (http).

Create the following Directory directives OUTSIDE THE (SSL) VIRTUAL HOST

<Directory "/var/www/html/ssl">
SSLRequireSSL
SSLOptions +StrictRequire
</Directory>

Why do these need to be outside the SSL virtual host? What is the best file to contain these? Why?

Restart Apache. Verify that your file can be accessed via SSL but not without SSL.

You should still be getting certificate warnings. If you have accepted the warnings from an earlier visit to your website, the warnings may no longer appear; simply close your browser and start again.

Certificate Authorities

Now we are going to create our own certificates and set up our own CA to sign them; doing so will let us import the certificate of the CA into browsers so that all of the certificates signed by that CA will no longer raise browser warnings.

We will be using this host to act as the equivalent of a trusted signer like VeriSign or Thawte. This does not need to occur on the same machine as the web server; in fact it can be done on a machine that is completely isolated from the network. In this example, we will use a separate machine just for this purpose.

The common name (CN) of the CA needs to be different from the common name of any machine for which you generate a certificate. For simplicity, we will suppose that

  • The DNS name of the web server is dollysisters.cosc.tu
  • The DNS name of the CA machine is dragonslanding.cosc.tu

Start by generating the private key for our CA:

[root@dragonslanding ~]# openssl genrsa -des3 -out /etc/pki/CA/private/ca.key 
4096
Generating RSA private key, 4096 bit long modulus
....................++
..............................................................................
...........++
e is 65537 (0x10001)
Enter pass phrase for /etc/pki/CA/private/ca.key:
Verifying - Enter pass phrase for /etc/pki/CA/private/ca.key:

Since this is the key for the CA, it should have a good passphrase.

Examine your CA key

[root@dragonslanding ~]# cat /etc/pki/CA/private/ca.key 
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,E41E99F323FB8DC6
<-- deleted material -->

Because this is the master key for the CA, it needs to be maximally protected. Notice that the default permissions on the directory /etc/pki/CA are 600

[root@dragonslanding ~]# ls -l /etc/pki/CA/ 
total 16
drwxr-xr-x. 2 root root 4096 Dec  7 16:48 certs
drwxr-xr-x. 2 root root 4096 Dec  7 16:48 crl
drwxr-xr-x. 2 root root 4096 Dec  7 16:48 newcerts
drwx------. 2 root root 4096 Mar  3 13:26 private

so that it is not readable by non-root users. If an outsider gains read access to this directory, they will be able to sign certificates just as you can; this would be bad.

Now we generate a certificate for our CA with our key. You will need the passphrase for your server key. You will need to enter some data for the CA certificate. Remember that the CN we will use for the CA is different than the name we will use for any web server certificates.

[root@dragonslanding ~]# openssl req -new -x509 -days 365 
-key /etc/pki/CA/private/ca.key -out /etc/pki/CA/certs/ca.crt
Enter pass phrase for /etc/pki/CA/private/ca.key:
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:Maryland
Locality Name (eg, city) [Default City]:Towson
Organization Name (eg, company) [Default Company Ltd]:Towson University
Organizational Unit Name (eg, section) []:Security Laboratory
Common Name (eg, your name or your server's hostname) []:dragonslanding.cosc.tu
Email Address []:   
Creating our certificate

Now we return to the web server (dollysisters.cosc.tu). The apache install has already created a key and certificate for the web server as localhost, stored in /etc/pki/tls/.

We will create a new server key by running

[root@dollysisters ~]# openssl genrsa 
-out /etc/pki/tls/private/dollysisters.key 4096
Generating RSA private key, 4096 bit long modulus
...................++
...................................................................
...........................++
e is 65537 (0x10001)

Note that we have stored the key in /etc/pki/tls/private and named it after the hostname.

You do not need a passphrase here; if you include one then you will need that passphrase whenever apache is started. Verify that you have a valid key

[root@dollysisters ~]# cat /etc/pki/tls/private/dollysisters.key 
-----BEGIN RSA PRIVATE KEY-----
MIIJKgIBAAKCAgEAzycxs5nEPb8mxNAZe7KokWNBcuViPAFZTbXo+zxJPQ2vb95j
<-- deleted material -->

If you want to see your public key, you can run

[root@dollysisters ~]# openssl rsa -in 
/etc/pki/tls/private/dollysisters.key -pubout
writing RSA key
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAzycxs5nEPb8mxNAZe7Ko
kWNBcuViPAFZTbXo+zxJPQ2vb95jr9UbpV5JbeAGiMKZT+W08NZju2s2F8JuqnaO
V4BJegD2TERWTB1yzLkbW00YixTPB7N8fZ5n9e6vMoO9XHO9M3/LpK1wFIQGv1ko
OCmz4ME/Y074licd60ELVFdzmONswwppoTpa3N/1FckQeoxMp8l1TXlxf1Q9JgsC
Susn/nD4pM2+J/R/KeOVuIRfP3hCpSr9mi0EDawWWslnfCUpgenFMMnlYAvnFpS+
g9zS/UmL2Am13hCeQvWd0kU7hXfEyObQAnlfzNtwhiBzsrBfXhV22o4F8Lz94PgJ
hVIXpo+jyqE2DAm4dyN2aS/uyTnINE1t27rMigdNMk7ZHI0QHNnwozuYTX+w1Q9m
dCcnePdJR20zecA7QWTlUwVxBhwZShuHAtA4u+hlziU7OQDCU0MMEU4nXl39UJZ5
W+3ysB3xGCf181vRTaSHaknqY5aP/LGl9AJ242zV6rnScK3nD8B16htxzCuqhL1f
QET60cEBq6rv6kowzXj8FpoY8iYeh7w01Dnu7ADn0Qv5kpC8EiF+V8NRkAh4/j4x
YOsHBmVmf52wdknF/gxsgqowSTgvZvJVn6tuasOmCN+wT9uEjgi/O3kfQuZVzSC7
vVkVWiJ5xkMNZ8wFK+qdS0MCAwEAAQ==
-----END PUBLIC KEY-----

Now we generate a certificate signing request (csr). Again you will need to answer some questions, but now about your server. The catch here is that the common name must match the name of your web site; otherwise your will see a domain mismatch error. Remember- this also must be different than the CN you selected for the CA.

[root@dollysisters ~]# openssl req -new 
-key /etc/pki/tls/private/dollysisters.key 
-out /etc/pki/tls/misc/dollysisters.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:Maryland
Locality Name (eg, city) [Default City]:Towson
Organization Name (eg, company) [Default Company Ltd]:Towson University
Organizational Unit Name (eg, section) []:Security Laboratory
Common Name (eg, your name or your server's hostname) []:dollysisters.cosc.tu
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Next, we send the certificate signing request over to our CA for its signature. In the lab this can be handled by simply copying the file on one virtual machine and pasting it in the other.

Now we use the CA (dragonslanding.cosc.tu) to sign the result:

[root@dragonslanding ~]# openssl x509 -req -days 365 -in ./dollysisters.csr 
-CA /etc/pki/CA/certs/ca.crt -CAkey /etc/pki/CA/private/ca.key 
-set_serial 01 -out /etc/pki/CA/newcerts/dollysisters.crt
Signature ok
subject=/C=US/ST=Maryland/L=Towson/O=Towson University/OU=Security Laboratory/CN=dollysisters.cosc.tu
Getting CA Private Key
Enter pass phrase for /etc/pki/CA/private/ca.key:

Now we copy the signed certificate back to the web server (dollysisters.cosc.tu) and store it in the file /etc/pki/tls/certs/dollysisters.crt.

Be sure to pay attention to file permissions. Copying files to/from a Windows host can leave the result executable and world writable. I wonder if that would be bad?

Now we wll tell the web server to use these keys. In the <VirtualHost> directive modify the following lines:

SSLCertificateFile /etc/pki/tls/certs/dollysisters.crt
SSLCertificateKeyFile /etc/pki/tls/private/dollysisters.key

Restart Apache. Check your logs (/var/log/messages and /var/logs/httpd/error_log) to be sure that Apache restarted cleanly.

View the SSL enabled page in a browser; you should still see an error saying that the issuer certificate is unknown. From the browser, view the certificates that are being used. You should see the certificate for the server, as well as the certificate of the CA.

Now we will import the key for the CA into our browser. Place the certificate ca.crt (NOT THE KEY) from the CA into a convenient location. In Firefox, navigate Edit → Preferences → Advanced → View Certificates.

Select Import, then select the file ca.crt; trust this CA to identify web sites.
When this is complete, you will see the entry for the CA in the list of trusted authorities:

When this is complete you should be able to visit the web site and view it without any SSL errors.

Notice that if you visit the web page without using the precise name in the certificate- by e.g. visiting https://192.168.1.67/ssl rather than https://dollysisters.cosc.tu/ssl then you will see that the certificate is only valid for the selected hostname.

Access Control

There are three important types of authentication

  • Basic Authentication
  • Digest Authentication
  • Form-based Authentication

The first two of these are part of the HTTP protocol (RFC 2167). The last is most common, and requires the web page(s) themselves to handle the authentication (via e.g. PHP). We will not cover this here.

Digest authentication, though an improvement on basic authentication, is not commonly supported by browsers; we will not cover this either.

Documentation on how to set up authentication can be found at http://localhost/manual/howto/auth.html

Here is a demonstration. For this purpose, we will put all of our configuration information in the main file (/etc/httpd/conf/httpd.conf) rather than in per-directory .htaccess files.

Create the directory /var/www/auth. We will store our authentication data here (outside of document root for the web server). Create the Apache password file, starting with a single user (HarryKing) as follows:

[root@dollysisters auth]# htpasswd -m -c /var/www/auth/passwd HarryKing
New password: 
Re-type new password: 
Adding password for user HarryKing

Note that the password for this user is manually entered.

Let’s view our password file:

[root@dollysisters auth]# cat /var/www/auth/passwd 
HarryKing:$apr1$AU1W1uXZ$0YOE9PBin0YMoHgzURs3t/

The -m option above told Apache to use the MD5 hash, which is exactly what we see in the file. Other options include DES (crypt()), SHA, and plaintext. More options for htpasswd can be found in the documentation at http://localhost/manual/programs/htpasswd.html

Let’s create a directory that will require password protection, and put a page inside.

[root@dollysisters html]# mkdir /var/www/html/secret
[root@dollysisters html]# vi /var/www/html/secret/index.html
[root@dollysisters html]# cat /var/www/html/secret/index.html 
<html>
This is a secret
</html>

At this point, we can still directly view the page at http://localhost/secret/. You should be able to explain why!

Now we update our configuration file (/etc/httpd/conf/httpd.conf) by adding the following:

<Directory "/var/www/html/secret">
AuthType Basic
AuthName "Restricted Files"
AuthUserFile /var/www/auth/passwd
Require valid-user
</Directory>

Restart Apache. Check your logs to verify that the restart was clean.

Now we are required to authenticate with a valid user/password combination from our file before we can view this page. Verify this!

Full documentation (including how to set up groups) can be found at http://localhost/manual/howto/auth.html.

It is also possible to use other than text files for authentication, including LDAP and databases. Consult the documentation for further details.

ModSecurity

Installation

ModSecurity is a web application firewall; it allows us to filter and analyze requests made of our Apache server at the application level. In this it differs from network firewalls, which focus on the packet level.

The installation of ModSecurity is going to require a fair bit of effort. To begin, there is no prepackaged binary available for CentOS, so we will compile it from source. Further, many of the required development libraries and packages are not pre-installed on the virtual machine provided to class.

The core requirements for installation include the packages

  • gcc
  • make
  • libxml2
  • libxml2-devel
  • httpd-devel
  • pcre-devel
  • curl-devel

However, each of those packages require other packages, and those require other packages, and so on. If our system was connected to the Internet, this can be handled by the package manager; we will do it by hand.

Connect the CD for the virtual machine to the installation .iso and navigate to the directory /media/CentOS_6.2_Final/Packages. From there, simply (hah!) run

[root@dollysisters Packages]# rpm -ivh ./libxml2-devel-2.7.6-4.el6.x86_64.rpm 
./httpd-devel-2.2.15-15.el6.centos.x86_64.rpm 
./libcurl-devel-7.19.7-26.el6_1.2.x86_64.rpm 
./pcre-devel-7.8-3.1.el6.x86_64.rpm 
./zlib-devel-1.2.3-27.el6.x86_64.rpm 
./apr-devel-1.3.9-3.el6_1.2.x86_64.rpm 
./apr-util-devel-1.3.9-3.el6_0.1.x86_64.rpm 
./libidn-devel-1.18-2.el6.x86_64.rpm 
./automake-1.11.1-1.2.el6.noarch.rpm 
./db4-devel-4.7.25-16.el6.x86_64.rpm 
./expat-devel-2.0.1-9.1.el6.x86_64.rpm 
./openldap-devel-2.4.23-20.el6.x86_64.rpm 
./autoconf-2.63-5.1.el6.noarch.rpm 
./db4-cxx-4.7.25-16.el6.x86_64.rpm 
./cyrus-sasl-devel-2.1.23-13.el6.x86_64.rpm 
Preparing...                ########################################### [100%]
   1:apr-devel              ########################################### [  7%]
   2:cyrus-sasl-devel       ########################################### [ 13%]
   3:openldap-devel         ########################################### [ 20%]
   4:db4-cxx                ########################################### [ 27%]
   5:db4-devel              ########################################### [ 33%]
   6:autoconf               ########################################### [ 40%]
   7:automake               ########################################### [ 47%]
   8:expat-devel            ########################################### [ 53%]
   9:apr-util-devel         ########################################### [ 60%]
  10:libidn-devel           ########################################### [ 67%]
  11:zlib-devel             ########################################### [ 73%]
  12:libxml2-devel          ########################################### [ 80%]
  13:libcurl-devel          ########################################### [ 87%]
  14:httpd-devel            ########################################### [ 93%]
  15:pcre-devel             ########################################### [100%]

Given what was already installed on the virtual machine, this will install all of the required packages (and their required packages, and their required packages and …)

Next, stop the web server (if it is running).

Grab the package modsecurity-apache_2.6.3.tar.gz either from the labshare or online. Uncompress the package in the usual fashion:

[root@dollysisters ~]# tar -xzvf ./modsecurity-apache_2.6.3.tar.gz 

and descend into the directory modsecurity-apache_2.6.3 that is created. From within that directory, run the configure script

[root@dollysisters modsecurity-apache_2.6.3]# ./configure 

This should complete without error if all of the required packages have been installed. To compile and install the program, run

[root@dollysisters bin]# make install

With the installation complete, add the lines

LoadFile /usr/lib64/libxml2.so
LoadFile /usr/lib64/liblua-5.1.so
LoadModule security2_module modules/mod_security2.so

to the file /etc/httpd/conf/httpd.conf after all of the other LoadModule lines (roughly line 202).

ModSecurity also requires the module unique_id_module; this is one of the modules not loaded by default. Uncomment out the line

LoadModule unique_id_module modules/mod_unique_id.so

We will also need a configuration file for ModSecurity. Fortunately it comes with a template that we can later customize; we move it to /etc/httpd/conf.d:

[root@dollysisters modsecurity-apache_2.6.3]# cp modsecurity.conf-recommended 
/etc/httpd/conf.d/modsecurity.conf

Because this file is located within the conf.d subdirectory, it will be loaded as part of the Apache startup. Validate your configuration and start the web server:

[root@dollysisters modsecurity-apache_2.6.3]# apachectl configtest
Syntax OK
[root@dollysisters modsecurity-apache_2.6.3]# /etc/init.d/httpd start
Starting httpd:                                            [  OK  ]

Note that Apache may restart even though ModSecurity did not, and ModSecurity may start, but not correctly, so be sure to check your logs. As an example, when preparing these notes I forgot to load the module unique_id_module. Despite that, I was able to verify my configuration file was correct and restart Apache; moreover the logs show no sign of trouble:

[Sun Mar 04 09:47:03 2012] [notice] caught SIGTERM, shutting down
[Sun Mar 04 09:47:03 2012] [notice] SELinux policy enabled; httpd running 
as context unconfined_u:system_r:httpd_t:s0
[Sun Mar 04 09:47:03 2012] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin/suexec)
[Sun Mar 04 09:47:04 2012] [notice] ModSecurity for Apache/2.6.3
(http://www.modsecurity.org/) configured.
[Sun Mar 04 09:47:04 2012] [notice] ModSecurity: APR compiled 
version="1.3.9"; loaded version="1.3.9"
[Sun Mar 04 09:47:04 2012] [notice] ModSecurity: PCRE compiled version="7.8";
loaded version="7.8 2008-09-05"
[Sun Mar 04 09:47:04 2012] [notice] ModSecurity: LIBXML compiled
version="2.7.6"
[Sun Mar 04 09:47:04 2012] [notice] Digest: generating secret for digest 
authentication ...
[Sun Mar 04 09:47:04 2012] [notice] Digest: done
[Sun Mar 04 09:47:04 2012] [notice] Apache/2.2.15 (Unix) DAV/2 
mod_nss/2.2.15 NSS/3.12.10.0 PHP/5.3.3 mod_ssl/2.2.15 OpenSSL/1.0.0-fips 
configured -- resuming normal operations

I then added a rule (we’ll talk about how to do this in a bit) and found that ModSecurity was not working. It was only when I went back and re-checked the logs that I later (notice the time stamps) found the entry

[Sun Mar 04 09:57:03 2012] [error] ModSecurity: ModSecurity requires 
mod_unique_id to be installed.

It turns out that ModSecurity only threw this log message when a request was made that matched the rule, not when the server was restarted.

This story has two morals- (1) Logs are your friend and (2) ModSecurity isn’t. If it is not behaving, be sure to read (and re-read) your logs.

Configuring ModSecurity

To configure ModSecurity, return to the file /etc/httpd/conf.d/modsecurity.conf. The first directive in that file

SecRuleEngine DetectionOnly

on line 7 tells ModSecurity that it should process all of its rules, but it should not take any action to modify or disrupt the connection. Documentation for this and for the other directives can be found in the doc subdirectory of the modsecurity-apache_2.6.3 directory. The most up to date documentation is available online.

Since this is not (yet) a production server, go ahead and change that setting to

SecRuleEngine On

Next, move to the end of the modsecurity.conf file and add the line

SecRule ARGS, "zzz" phase:1,log,deny,status:503

We are going to use this rule solely for testing. It tells ModSecurity to look at the arguments of the request for the string "zzz"; if it appears then log the request, and return a service temporarily unavailable (503) error.

With these changes made, restart Apache. Then from a browser on another system visit the web page of your server and include the text "zzz" as one of the parameters. Since GET parameters are specified in the URL, we can simply visit:

If the system is configured correctly, you will get the 503 error.

Return to the modsecurity.conf file.

The configuration directives in the Request body handling and the Response body handling sections (lines 10-95, 96-122) can be left in their default state.

If your web server is used solely as a web server, then you can keep the Filesystem configuration variables (lines 133-139) in their default state; i.e. using /tmp as the temporary directory for ModSecurity. If multiple users are going to be present on the system though, then you may wish to consider an alternate structure. You could, for example, create the directory /var/www/modsecurity, set its permissions and owner appropriately (e.g. owner apache:apache, permissions 700) and specify that location here.

The File uploads handling configuration (lines 139-159) determines what should happen to uploaded files after ModSecurity finished processing them. To keep things simple, let’s agree to drop them and so set

SecUploadKeepFiles Off

The Debug log is, oddly enough used to store debugging messages from ModSecurity. The commented out location in the configuration file (/opt/modsecurity/var/log/debug.log) does not even exist. Start by creating a file- say /var/log/modsec_debug.log, and modify the directive:

SecDebugLog /var/log/modsec_debug.log

The SecDebugLogLevel sets the level amount of detail to be contained in that file.

  • Level 0: No logging
  • Level 1: Fatal errors
  • Level 2: Warnings
  • Level 3: Notices
  • Level 4: Informational
  • Level 5: Details
  • Level 9: Everything

Though useful, be careful with this setting. These are debugging messages, and can be used to debug rules that you create. On the other hand, they generate a lot of data. At level 5 with just the one rule, the debug log ends up with 24 entries for a single request. The recommendation is to use this log sparingly; certainly not higher than level 3 without cause.

ModSecurity began as a project to provide full HTTP transactional logs; this is done through the Audit logs (lines 171-192). They are able to log all parts of a web request, both inbound to the server and outbound from the server.

The default settings have ModSecurity only log those requests that generate a server error (other than 404), though the settings can be modified to record every request.

Each request is broken into parts, including

  • The request header
  • The request body
  • The response header
  • The response body

Some, all, or none of these parts can be recorded; this is set via SecAuditLogParts; see the documentation for more detail.

A typical audit log entry looks like

--c112c62f-A--
[04/Mar/2012:10:54:46 --0800] T1O6dn8AAAEAABmTNJcAAAAA 192.168.1.60 35822 192.168.1.67 80
--c112c62f-B--
GET /?a=2zzz HTTP/1.1
Host: dollysisters.cosc.tu
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) 
Gecko/20111109 CentOS/3.6.24-3.el6.centos Firefox/3.6.24
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Cache-Control: max-age=0

--c112c62f-F--
HTTP/1.1 503 Service Temporarily Unavailable
Content-Length: 409
Connection: close
Content-Type: text/html; charset=iso-8859-1

--c112c62f-E--
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Temporarily Unavailable</title>
</head><body>
<h1>Service Temporarily Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
<hr>
<address>Apache/2.2.15 (CentOS) Server at dollysisters.cosc.tu Port 
80</address>
<body></html>

--c112c62f-H--
Message: Access denied with code 503 (phase 1). Pattern match "zzz" 
at ARGS:a. [file "/etc/httpd/conf.d/modsecurity.conf"] [line "207"]
Action: Intercepted (phase 1)
Stopwatch: 1330887286434508 1635 (- - -)
Stopwatch2: 1330887286434508 1635; combined=65, p1=58, p2=0, p3=0, 
p4=0, p5=7, sr=0, sw=0, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.6.3 (http://www.modsecurity.org/).
Server: Apache/2.2.15 (CentOS)

--c112c62f-Z--

Here you can see the headers and the body for both the request and the response. They are coded by the particular request (c1112c62f) and the part (e.g. A, B, … H, Z)

By default, the audit log is stored in the file /var/log/modsec_audit.log, and there is no need to modify this from the default.

ModSecurity Core Rules

At this point, the installation of ModSecurity is complete. However, this does not mean that ModSecurity is doing much. In fact, so far all it is doing is throwing an alert whenever the text "zzz" appears in the arguments of a request- and we should probably remove that test rule too.

Though the ModSecurity module is now inspecting the traffic coming to and from the web server, we really have not given it much information on what sort of traffic should be blocked or what should be allowed. To that end, we turn to OWASP and the OWASP ModSecurity Core Rule Set. These are a set of rules for ModSecurity that will let us detect and block a number of common attacks on web applications.

Begin by grabbing the core rule set, either from the labshare or online. Unpack the package in a convenient location and move the result to /etc/httpd/conf/modsecurity_crs.

[root@dollysisters ~]# tar -xzvf ./modsecurity-crs_2.2.3.tar.gz 
[root@dollysisters ~]# mv modsecurity-crs_2.2.3 /etc/httpd/conf/modsecurity_crs

The rule set comes with an example configuration file; we rename that file:

[root@dollysisters modsecurity_crs]# cp modsecurity_crs_10_config.conf.example
modsecurity_crs_10_config.conf

The rule set currently comes with four classes of rules:

  • Core rules
  • Optional rules
  • Experimental rules
  • TrustWave SpiderLab rules

The latter of these are a free certified rule set aimed at attacks on common web applications.

We enable the core rules and the SpiderLab rules by adding the following

#ModSecurity Includes
Include conf/modsecurity_crs/modsecurity_crs_10_config.conf
Include conf/modsecurity_crs/base_rules/*.conf
Include conf/modsecurity_crs/slr_rules/*.conf

to our /etc/httpd/conf/httpd.conf file.

With this complete, simply restart Apache.

Now suppose that an attacker is trying to exploit a vulnerable web application; in many such vulnerabilities, the attacker needs to null terminate a string. Suppose for simplicity that the string is being passed as a GET parameter; the usual way to null terminate a GET parameter is to encode the null using a URL in the form

http://dollysisters.cosc.tu/?a=string%00

Here we see the null byte to the end of the GET parameter a.

If a user makes a non-malicious request of our ModSecurity protected web server, they get the correct response:

On the other hand, if they pass a null at the end of the GET parameter, they end up at a default page:

and they end up in the modsec_audit.log

--79d2535d-A--
[04/Mar/2012:12:16:40 --0800] T1PNqH8AAAEAABtxQjEAAAAB 192.168.1.60 36011
192.168.1.67 80
--79d2535d-B--
GET /?a=string%00 HTTP/1.1
Host: dollysisters.cosc.tu
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) 
Gecko/20111109 CentOS/3.6.24-3.el6.centos Firefox/3.6.24
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive

--79d2535d-F--
HTTP/1.1 403 Forbidden
Accept-Ranges: bytes
Content-Length: 5039
Connection: close
Content-Type: text/html; charset=UTF-8

--79d2535d-H--
Message: Unable to retrieve collection (name "global", key "global"). 
Use SecDataDir to define data directory first.
Message: Unable to retrieve collection (name "ip", key
"192.168.1.60_e21c66ac2b509d0e2766e2f938b8f28a4cf78521"). Use SecDataDir 
to define data directory first.
Message: Access denied with code 403 (phase 2). Found 1 byte(s) in 
ARGS:a outside range: 1-255. [file
"/etc/httpd/conf/modsecurity_crs/base_rules/modsecurity_crs_20_protocol_
violations.conf"] [line "353"] [id "960901"] [rev "2.2.3"] [msg "Invalid 
character in request"] [severity "WARNING"] [tag 
"PROTOCOL_VIOLATION/EVASION"] [tag "WASCTC/WASC-28"] [tag 
"OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/RE8"] [tag "PCI/6.5.2"] [tag 
"http://i-technica.com/whitestuff/asciichart.html"]
Action: Intercepted (phase 2)
Stopwatch: 1330892200423870 11420 (- - -)
Stopwatch2: 1330892200423870 11420; combined=866, p1=697, p2=126, p3=0, 
p4=0, p5=43, sr=383, sw=0, l=0, gc=0
Producer: ModSecurity for Apache/2.6.3 (http://www.modsecurity.org/); core
ruleset/2.2.3.
Server: Apache/2.2.15 (CentOS)

--79d2535d-Z--

Multiple IP Addresses

You may want to set up a host with multiple external IP addresses. These separate IP addresses can also have their own separate names. Since most services allow you to sepcify the IP address(es) on which the service listens, one physical host can appear to the network to be a range.

To set this up in, for example, CentOS, you can modify the network configuration scripts in /etc/sysconfig/network-scripts.

Create a new file, called ifcfg-eth0:0; copy it from your existing /etc/sysconfig/network-scripts/ifcfg-eth0. Modify the Device name in that file to read eth0:0. Modify the static IP address to whatever value you wish. You cannot use DHCP, but you can delete the gateway specification. Do not modify the hardware MAC address. The resulting file will look something like the following:

[root@dollysisters ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0:0
DEVICE=eth0:0
IPADDR=192.168.1.68
NETMASK=255.255.255.0
ONBOOT=yes
HWADDR="00:0C:29:5D:79:9E"

After a reboot, you will see both interfaces in ifconfig:

[root@dollysisters ~]# ifconfig 
eth0      Link encap:Ethernet  HWaddr 00:0C:29:5D:79:9E  
          inet addr:192.168.1.67  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe5d:799e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:125 errors:0 dropped:0 overruns:0 frame:0
          TX packets:60 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:19577 (19.1 KiB)  TX bytes:8518 (8.3 KiB)

eth0:0    Link encap:Ethernet  HWaddr 00:0C:29:5D:79:9E  
          inet addr:192.168.1.68  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:14 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:984 (984.0 b)  TX bytes:984 (984.0 b)

This process is called setting an alias for an interface. A similar process can be used on other Linux distributions.

Because your host now has two IP addresses, it can also have two host names, provided the DNS server is appropriately configured.

  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 177 other followers

%d bloggers like this: