AWS: Ruby on Rails Deployment Part 1: Nginx Installation and Configuration

Over the course of this series of articles, I will cover the build and configuration of an Amazon EC2 Instance capable of serving Ruby on Rails applications. The series will cover the build and installation of Nginx from source, virtual host and proxy configuration within Nginx, installation of Ruby and RubyGems, installation of the Rails and Thin gems, and the deployment of a set of clustered Thin workers. I chose Nginx over Apache HTTPD as it is renowned for both performing very well as a reverse proxy as well as serving static content whilst having a very low memory footprint. Plus, I’m always interested in looking at “alternative” software solutions to common problems.

Read my article around EC2 instance management via the ec2-api-tools if you’d like to provision your instance(s) via the command line, otherwise just provision your instance(s) via the EC2 Management Console. This article presumes that you have an instance running and ready to go. I used ami-08df4961 (which is Ubuntu 12.10 i386 Server, EBS-backed). I’d use a RHEL instance but they are not eligible for the free tier due to licensing, plus the Ubuntu instances are very well supported by Canonical.

Continue reading →

ZFS Part 5: ZFS Clones and Sending/Receiving ZFS Data

A ZFS Clone is a read-write clone of a filesystem created from a snapshot. It still refers to the snapshot it has been created from, but allows us to make changes. We cannot remove the origin snapshot whilst the clone is in use, unless we promote it. These concepts will become clear during the examples.

Continue reading →

OpenDJ: Installation and Multi-Master Replication Configuration

OpenDJ is a fork of the OpenDS directory server, with OpenDS originally being developed by Sun. Sun went its own way with JDS, and OpenDS was developed for a while by the community. ForgeRock came along and forked the stale project off (the last OpenDS release was back in 2010) and have been actively developing and since then have been releasing builds as OpenDJ. OpenDJ forms part of the complete ForgeRock open-identity stack, fully supported, and is a very viable alternative to the Sun/Oracle IDM stack. OpenDS, and therefore OpenDJ, are pure Java implementations.

In this article, I will install and configure OpenDJ on two nodes running Oracle Enterprise Linux 5.7 x86_64. OpenDJ will be configured with multi-master replication between both nodes. The same procedure should work on any RHEL 5.x or 6.x derivative. Node names are foo-0001prjenv and foo-0002prjenv.

You should obtain the latest versions of all software for your platform.

Continue reading →

NSD: How to Implement Master and Slave DNS Servers

NSD is a name server implementation developed and maintained by NLnet Labs in cooperation with RIPE. NSD is an authoritative-only DNS implementation, and is memory efficient, secure and fairly straightforward.

NSD can start up quickly, as all zone information is compiled into an efficient binary format prior the the NSD daemon reading it. A few of the root nameservers use NSD, as well as the .se ccTLD. NSD has a proven history of being robust and can meet the demands of the highest-traffic DNS requirements known.

You can read more about NSD over at the project page (http://www.nlnetlabs.nl/projects/nsd/) so I won’t dwell on the preamble. My NSD implementation will run a master instance on server gooby and a slave instance on server dolan. There were a few pitfalls along the way, as this was my first NSD implementation. Unfortunately, there don’t seem to be many NSD resources available on the internet, so I’ll include those pitfalls in this article and hopefully save you some of the time/pain/strace/bash -x that I had to indulge myself in. Both gooby and dolan are running a minimal package install of CentOS 6.3 x86_64.

Continue reading →

PowerDNS 3.2: First Steps

I have been an avid PowerDNS Authoritative Server (hereafter referred to as PowerDNS - there is also a separate Recursor available that we shall ignore for now - both available at http://www.powerdns.com) user since early in the 2.x series of releases. I replaced a global BIND infrastructure with PowerDNS for many reasons - instant provisioning to an easily replicated MySQL backend being the main one. PowerDNS is also RFC-compliant, powerful, and reliable.

The infrastructure I commissioned served well over 200,000 zones across three nameserver sites - each site receiving well over 100,000,000 queries per day. Two servers at each site, each with their own MySQL backend, replicated to from a hidden MySQL master to which we provisioned, handled this load with ease. Of course, this will depend entirely on the server specifications you use to host PowerDNS. PowerDNS offers support for multiple backends, however MySQL suits my needs well - I’m familiar with it, it’s more suited to DNS provisioning than LDAP (IMHO) and supports native replication (unlike PostgreSQL).

Rather than hacking a provisioning solution around BIND 9, moving to PowerDNS provided a technical advantage as well as a business advantage - customers could have their DNS data provisioned near-instantly - something that BIND 9 with a large number of zones and a cron’d rndc reconfig/reload would not achieve. PowerDNS 3.x introduces support for DNSSEC, something PowerDNS 2.x didn’t have - so it’s time to move to PowerDNS 3.x where possible.

I will use this article to walk through an installation of PowerDNS 3.2 from source on CentOS 6.3, perform basic configuration, load a basic zone, and serve the zone data authoritatively. This article will only scrape the surface of what PowerDNS has to offer, and further articles will be written in due course to cover interesting concepts in finer detail.

Have a good read over the manual available on the PowerDNS documentation site (http://doc.powerdns.com) too.

Continue reading →

Highly-Available Load Balancing of Apache Tomcat using HAProxy, stunnel and keepalived

This article will describe the process required to set up a highly-available SSL-enabled load balancer using HAProxy, stunnel and keepalived to front a pair of Apache Tomcat instances.

The configuration will start off simple, and extend and grow as more functionality is added. First, a session-aware HAProxy load balancer is configured. Next, Tomcat clustering is configured with session replication and the responsibility for maintaining session state is moved to Tomcat. Then, keepalived is added - providing a failover IP between HAProxy instances running on two nodes. The solution is then fully HA. To complete the article, SSL support will be enabled by way of adding stunnel in front of the HAProxy instances, and a few cleanup tasks performed.

Faithful dolan and gooby will be used for this in my lab environment. Each of the servers is running CentOS 6.3 x86_64 and already has a Tomcat instance installed to /usr/local/tomcat7 (running as the tomcat user).

As you can see, I’ve also reserved an IP address for our floating IP (or Virtual IP - VIP).

The end goal is to have dolan and gooby both running Tomcat as before, also running their own HAProxy and stunnel instances but also running a failover VIP provided by keepalived.

Continue reading →

OpenLDAP Multi-Master Replication

This article will cover the installation of two OpenLDAP instances on different nodes. OpenLDAP will be configured to provide multi-master replication. Writes should still be sent to a single server, but both can be used for directory reads. In the event of failure of the write node, a load balancer or similar solution could fail writes over to the other node. OpenLDAP multi-master replication to me doesn’t seem like true multi-master replication, rather an active-active read LDAP database with hot standby of a write database. In this author’s opinion, (Oracle) Sun JDS and Forgerock OpenDJ offer far superior replication abilities.

This article will not cover OpenLDAP basics which you can already find within the administration guide available from OpenLDAP. Review the documentation available there before proceeding with this tutorial.

Continue reading →

ZFS Part 4: ZFS Snapshots

Snapshots are another piece of awesome functionality built right into ZFS. Essentially, snapshots are a read-only picture of a filesystem at a particular point-in-time. You can use these snapshots to perform incremental backups of filesystems (sending them to remote systems, too), create filesystem clones, create pre-upgrade backups prior to working with new software on a filesystem, and so on.

The snapshot will, initially, only refer to the files on the parent ZFS dataset from which they were created and not consume any space. They will only start to consume space once the data on the original dataset is changed. The snapshot will refer to these blocks and will not free them, thus the snapshot will start consuming space within the pool. The files on the snapshot can be accessed and read via standard UNIX tools (once you know where to look).

Continue reading →

DECNet Phase IV Fails to Install on OpenVMS 7.3

During installation of OpenVMS 7.3 on a simulated VAXserver 3900 under SIMH, I selected the installation of DECnet Phase IV as follows:

The attempted installation of DECnet Phase IV fails, and the following system messages are generated:

First, let’s mount our OpenVMS media and see if we can locate the appropriate kit manually:

OK, we’ve found our kit. Let’s install it:

Confirm that the .PCSI file is in the current directory:

And install the kit:

Excellent - DECNet Phase IV is now correctly installed. You can proceed with applying your DECNet license PAKs (the LICENSE command), and configuration of DECNet as appropriate (@SYS$MANAGER:NETCONFIG.COM).

BIND 10: Authoritative DNS Server

Note: This article was written alongside BIND 10 1.0.0-beta. See this article which covers the major differences to be aware of between 1.0.0-beta, and 1.0.0-rc.

In the previous article in the BIND 10 series, we saw how to build and install BIND 10, start it up (in the background), get it running as a designated user, connect to the command channel, and issue a command to shut BIND 10 down. In this article, we will see how BIND 10 needs to be configured to act as an authoritative DNS server. Further articles will deal with recursion, the DHCP components, and more as I investigate BIND 10 in more detail. I’m learning as I write too, so bear with me - BIND 10 is very new software, and is still in Beta stages of development - thus a lot of BIND 10 is still incomplete, or even where complete - incoherent - especially to a BIND 9 veteran like myself!

If you haven’t read the BIND 10: First Steps article, I suggest you go and do that now.

We will use bindctl to interact with the BIND 10 command channel. I’ve had a bit more time to dig into bindctl and what it actually does (and where it stores its configuration) so we will digress a little at first, and take a look at that.

Continue reading →