We’re breaking our monolithic application into a series of smaller services that a much lighter-weight Rails application will then utilize. These services will be (initially) sitting on the same machine behind an apache load balancer. The front application will be talking SSL to the load balancer. This post is essentially how to wire this all together (so it can be tested in a configuration similar to production)

Before starting, here are the assumptions I am going to be working from:

  • The front application is currently able to talk directly to each of the services directly (no proxy or clustering)
  • Rails is being served by mongrel. Sorry Monorail or Passenger kids; you are too bleeding edge for me
  • This was written on a mac so some of the security conventions (like sudo for lots of stuff) might be OS quirks

Part 1: Generate SSL Certificates

In production, any customer port that is secured should likely be done through a commercial Certificate Authority (CA) as they have their Root Certificates installed in browsers by default (a perk they pay dearly for). But given that this is being done in the context of testing and the services will only only be used by clients of our creation we can use a pretend CA. The one that is in the copy of OpenSSL that is on this machine is called demoCA. Let’s set it up.

Setup your CA
$ mkdir ~/certificates
$ cd ~/certificates
$ mkdir demoCA
$ mkdir demoCA/certs
$ echo 00 > demoCA/serial
$ touch demoCA/index.txt

Now that you have a bit of infrastructure taken care of, you need to make the CA’s key

$ openssl genrsa -des3 -out ca.key 1024

Then create a new request and sign it with the key in one shot

$ openssl req -new -x509 -days 365 -key ca.key -out ca.crt

Now that we have a CA you can make the request for your service. But first it needs a key too.

$ openssl genrsa -out s1.key 1024

(alternatively you could have added a -des3 to password protect the key, but it is a pain. don’t do it)

Here is the actual request. It is very important that when it prompts you for the ‘Common Name’ (CN) that you put in the host name of the machine you are trying to access. For instance, adam.zerofootprint.net. Crypto is all about trust and one of the checks is that the hostname matches the cn.

$ openssl req -new -key s1.key -out s1.csr

And sign it using our CA

$ openssl ca -in s1.csr -out s1.pem -keyfile ca.key -cert ca.crt

If you put a password on your server’s keyfile, you should remove it. It’s not necessarily as secure anymore, but…

$ cp s1.key s1.key.orig
$ openssl rsa -in s1.key.orig -out s1.key

Repeat the creation of certificates for the amount of services you have making sure to change the name of the files things are saved into.

Part 2: Setup Virtual Servers
I really don’t like how Apache is configured on the Mac by default, so this next section will likely be redundant to Ubuntu or other nicely configured servers.

Everything in Apache for the last couple years has been setup as a Virtual Server. Even the default port (80) is setup in that manner. We’re going to pretend that there are 2 services going to be hosted in this setup. Infrastructure again.

$ cd /etc/apache2
$ sudo mkdir sites-available
$ sudo mkdir sites-enabed
$ cd sites-available

In the sites-available directory you put in the actual virtual host definitions. Here is the virtual host definition I started with for one of the services. I also have one for port 7501.

Listen 7500
NameVirtualHost *:7500

<VirtualHost *:7500>
DocumentRoot "/Library/WebServer/Documents/s1"
# you need a different ServerName for each host
ServerName s1.your.site
ServerAdmin you@example.com
ErrorLog "/private/var/log/apache2/s1-ssl-error.log"

</VirtualHost>

In order to have apache pick up these hosts you need to include them in your main config file. On the mac it is httpd.conf.

Include /etc/apache2/sites-enabled/*

This of course won’t work without a bit of magic. I’m only showing the first service, but you have to do this for each one.

$ cd /etc/apache/sites-enabled
$ ln -s ../sites-enabled/s1 s1

Restart apache and make sure that you can get to all your virtual hosts. They are still running in the clear over http.

Part 3: SSL-ize Virtual Servers
The end goal of course is have communications to these virtual servers to be secured so now we actually turn it on. Again, some infrastructure.

$ cd /etc/apache2
$ sudo mkdir certificates
$ sudo cp ~/certificates/s1.key .
$ sudo cp ~/certificates/s1.pem .

Repeat for each of your service keys that you made in Part 1.

Now modify your virtual hosts

Listen 7500
NameVirtualHost *:7500

<VirtualHost *:7500>
DocumentRoot "/Library/WebServer/Documents/s1"
# you need a different ServerName for each host
ServerName s1.your.site
ServerAdmin you@example.com
ErrorLog "/private/var/log/apache2/s1-ssl-error.log"

# these were all the defaults from the stock mac install; tune as necessary/desired
SSLEngine on
SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
SSLCertificateFile "/private/etc/apache2/certificates/s1.pem"
SSLCertificateKeyFile "/private/etc/apache2/certificates/s1.key"

BrowserMatch ".*MSIE.*" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0

CustomLog "/private/var/log/apache2/ssl_request_log" \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

</VirtualHost>

Again, restart apache and see that going to your ports over https works and you have the correct certificate being served (view the certificate and check the CN)

Part 4: Contacting the Nodes
I’m going to skip how you setup a mongrel cluster as there are a tonne of sites that deal with that already. In this example I’m going to have a 2 node cluster.

We’re going to use the same convention for the cluster nodes as we used for the virtual hosts.

$ cd /etc/apache2
$ sudo mkdir nodes-available
$ sudo mkdir nodes-enabled
$ cd nodes-available

In the nodes-available you want to define the members of each node. Each of our services is going to have only 1 node in this example. And each member is going to listen on localhost.

BalancerMember http://127.0.0.1:8000
BalancerMember http://127.0.0.1:8001

Now you might be a bit alarmed that it is using http which is somewhat ironic given the whole point of this is to be more secure, but if someone has the ability to sniff localhost then you are screwed already. Naturally, each member needs to have its own ports and that the ports are correct.

In order to redirect clients around to the various nodes we need to do a bit of modification to our virtual hosts file we created in step 2 and added ssl stuff to in 3. Specifically the Proxy section and the rewrite rules.

Listen 7500
NameVirtualHost *:7500

<Proxy balancer://s1_cluster>
  Include "/etc/apache2/nodes-enabled/s1*"
</Proxy>

<VirtualHost *:7500>
  # various SSL settings including certificates for this port go here

  RewriteEngine On
  RewriteRule ^/(.*)$ balancer://s1_cluster%{REQUEST_URI} [P,QSA,L]
</VirtualHost>

That’s all there is to it. Now you will be able to run multiple services on the same host with proper ssl certificates. Once a service has outgrown its ability to share space with the other services you can just move the certificate to a separate machine and tweak the dns to point to the new location (since the CN is the machine name). No service reconfiguration necessary.

Hope this saves someone some thinking.