Tag Archives: CentOS

Configuring Interface Bonding on CentOS/RHEL/OEL 6.x

In my previous article I wrote about configuring configuring network interface bonding under Debian Wheezy. Here, I’ll briefly outline the steps required to get the same configuration running under recent RHEL-flavoured distributions - namely CentOS 6.4 in my case.

I will be bonding eth0 and eth1 into a bond named bond0. Ensure that you’re connected to your host via a console. I’ll be using active-backup (i.e. failover) bonding, but there are other options available - see the Debian article for links to reference material for those.

First, create the ifcfg-bond0 configuration file:

Substitute relevant values as appropriate for your setup. Next, edit/create the ifcfg-eth{0,1} files. Note that these are created as slave interfaces (SLAVE=yes) with bond0 as the master interface (MASTER=bond0):

Note - that if NM_CONTROLLED is set, you should strictly define your HWADDR entries too at this step, for each interface. Configure the bonding module. miimon is the MII link monitoring frequency in milliseconds, {down,up}delay are the times, in milliseconds, to wait before disabling or enabling an interface in the bond (to safeguard against flapping), and should be a multiple of the miimon value. Note that /etc/modprobe.conf is deprecated in CentOS 6.x so an appropriate file should be created under /etc/modprobe.d - in our case, bonding.conf:

To test, manually load the module (and appropriate options - I see many tutorials with a simple modprobe bonding here - you’ll end up with the default bonding mode which is round-robin - not what we want):

And restart networking:

Verify that all is well with ifconfig -a, or more suitably a cat on /proc/net/bonding/bond0:

Reboot the host at the earliest opportunity to verify that all is well after a reboot.

 

 

Securing and Hardening the Linux Operating System

Any application is only as secure as the operating system hosting it. Whilst most servers are protected behind firewalls with the main avenue of exploit being the application, if an application is exploited and the user did manage to obtain server access, we need to minimise the damage that can be done to the exploited machine and others. Running hardened systems is a requirement of certain data security standards such as PCI-DSS (PCI Application Data Security Standard), as well as mandated by many organisations. Plus, it is just good practice. Out-of-the-box, most modern OSes are configured to be secure. There is still a great deal we can do, however, to further harden the operating system to various vectors of attack.

This article will only cover the core aspects of Linux OS lockdown, and will focus on CentOS 6.4. Most of the concepts here should be able to be ported to other flavours, using the appropriate commands for your OS.

For further detail, consult the security benchmarks available at http://cisecurity.org. You should employ multiple layers of security within your architecture – for example, using network-based firewalls as well as host firewalls, writing secure code, ensuring that all aspects of the application stack are patched and up-to-date, etc. Hardening the OS is just one more layer of security.

Continue reading →

Puppet Module: security::tcpwrappers with Hiera

Module: security::tcpwrappers

Purpose: This module configures TCP Wrappers on CentOS 6 and Solaris 10 hosts with Hiera.

Notes: This module does some pretty fancy things. It uses Hiera to provide lookup for the $hostsallow and $hostsdeny variables, and interfaces with inetadm and svccfg on Solaris. Let’s look at hiera.yaml first.

File: /etc/puppet/hiera.yaml

As you can see, we first check the %{::clientcert}, then the %{::operatingsystem} before falling back to common. So, essentially you have hostname-specific control if you need it.

Under /etc/puppet/hieradata, I have the following:

Host centosa.local would use centosa.local.yaml (due to %{::clientcert} in the hierarchy) and pull the values in for security::tcpwrappers::hostsallow and security::tcpwrappers::hostsdeny from that file. Host centosb.local would fall through to common.yaml (unless there was a %{::clientcert}.yaml or Centos.yaml), and a Solaris host would use Solaris.yaml.

File: security/manifests/tcpwrappers.pp

Notes: Uses Hiera to copy in appropriate files. On Solaris, it configures inetd-controlled services to use TCP Wrappers via inetadm, and enables TCP Wrappers for the RPC portmapping service via svccfg.

 

 

Secure MySQL Replication over SSL

MySQL is a popular open-source relational-database management system. One of its core features is replication, and in this article I will be showing how to configure a master and slave MySQL instance, and then configure replication from master to slave over SSL. Encryption will help protect the replication from snooping. This type of replication has many uses, for example: disaster-recovery scenarios whereby the slave can be switched to a master role in the case of a master outage, for performance where all reads can take place on the slave with writes and updates occurring on the master, and so on. Replication can be configured without encryption, but encrypting with SSL is preferred as part of a defence-in-depth strategy - it’s an extra layer of security.

This article already presumes a good working knowledge of MySQL. The master server is centosa with IP address 10.1.1.150, and is running a minimal installation of CentOS 6.4 x86_64. The slave, centosb, is running the same OS and has IP address 10.1.1.151. MySQL will be installed from the latest current stable RPMs available at dev.mysql.com, rather than using the upstream versions. The latest stable version available at the time of writing is 5.6.14.

This article will cover the configuration of an SSL-encrypted replicated environment from scratch - it does not cover the migration of an existing replicated configuration to an SSL-encrypted replicated configuration, or the migration of any existing data to a new slave.

Continue reading →

SELinux: Allowing httpd to Listen on Non-Standard Ports

If you attempt to have httpd listen on a non-standard port (we’ll get to what’s “standard” and what isn’t in a moment) on an SELinux enabled host, SELinux will deny the request. Let’s try it out (on a CentOS 6.4 box):

Kaboom! As you can see, the error message is quite clear “(13)Permission denied: make_sock: could not bind to address”. Let’s take a look at the audit logs and see what’s going on:

There are two sets of messages there, one for the IPv4 bind, the other for the IPv6 bind. The AVC message actually makes it clear what’s going on:

type=AVC msg=audit(1371477046.382:20195): avc: denied { name_bind } for pid=16429 comm="httpd" src=8585 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=system_u:object_r:port_t:s0 tclass=tcp_socket

You can see that the violation is from source context (scontext) unconfined_u:system_r:httpd_t which is the httpd process, and the target context (tcontext) is system_u:object_r:port_t with target class (tclass) tcp_socket. The denied port (8585 in this case) is also logged in the src field.

SELinux will only allow httpd to bind to ports of type http_port_t. Here, we can see that httpd is attempting to bind to a port of type port_t - which is going to be denied by the policy. We need to define port 8585 as a port of type http_port_t. semanage is the tool for that - and you’ll need the policycoreutils-python package installed:

First, let’s look at which ports are currently defined as type http_port_t:

And add our desired listen port, 8585/tcp to the list:

Looks good. Fire up httpd:

And test with nc or similar:

 

rsyslog - “Could not open dynamic file - discarding message” errors on CentOS 6

Whilst configuring a central syslog server using rsyslog on CentOS 6, I was experiencing issues with dynamic log filenames being created. I had created a new filesystem, mounted at /var/syslog, for my logs, and configured rsyslog.conf with the following:

However, the dynamic logs were not being created, and instead the following error message was observed in the local /var/log/messages file (mars being the hostname):

A quick check of the SELinux context found the issue:

chcon to the rescue, referencing /var/log:

A restart of rsyslog later, and we were in business:

Note: it’s worth noting here that I updated my dynamically created file rule to be as follows (including the date too) as one huge file per host is not very useful:

 

Running Puppet Master under Apache and Passenger - CentOS 6.4

I have been running my puppetmaster using the embedded WEBrick server for a while. I decided it was time to migrate to something a little more robust - namely Apache and Passenger. I loosely followed the documentation available on the Puppet site, although that covers Passenger 3.0.x and I’m using 4.0.x, and the supplied Apache configuration does not work. There were also a few other changes I had to make along the way to suit my configuration requirements. My puppetmaster is running CentOS 6.4.

Continue reading →

Adding Puppet DB to a CentOS 6.4-based puppetmaster

After configuring my puppetmaster to run on Passenger on Apache, the next step was to add Puppet DB so that I could take advantage of exported resources, as well as have a central store of facts/catalogs that I could query. I am configuring Puppet DB to run on the same host as my puppetmaster, on CentOS 6.4.

Whilst you can use the embedded HSQLDB database, I opted for the scale-out offered by a backend PostgreSQL database (which is recommended by Puppet Labs for deployments of more than 100 nodes). MySQL is not supported, as it doesn’t support recursive queries.

I installed PostgreSQL using my postgresql::server class. The class is shown below (you can see it includes postgres, which does nothing more than install the PostgreSQL client package). Essentially, it covers the installation of the postgresql-server package on the host, initialisation tasks, ensuring an appropriate pg_hba.conf is installed (for example, replacing localhost ident authentication methods with password), registering the service, and starting PostgreSQL:

To install, it was as simple as:

Next, I created a PostgreSQL user and database:

The user is created so that it cannot create databases (-D), or roles (-R) and doesn’t have superuser privileges (-S) - it’ll prompt for a password (-P). Let’s assume a password of “s3cr3tp@ss” has been used. The database is created and owned (-O) by the puppetdb user.

Access to the database can then be tested:

Next, install the puppetdb and puppetdb-terminus packages. I installed from the Puppet Labs yum repository:

Configure /etc/puppetdb/conf.d/database.ini as appropriate (it ships configured for HSQLDB):

Create /etc/puppet/puppetdb.conf, as appropriate for the Jetty configuration on the host. You can check the Jetty configuration under /etc/puppetdb/conf.d/jetty.ini. Here’s puppetdb.conf (my puppetmaster runs on sun.local):

Create /etc/puppet/routes.yaml:

Finally, update /etc/puppet/puppet.conf:

Once all steps are complete, restart your puppetmaster. As I’m running under Passenger:

Run a puppet agent --test from once of your nodes (or wait for your scheduled runs):

As you can see, I ran from venus.local. Let’s check that the data is being stored in puppetdb - by the time I got there three more nodes had already had an agent run:

Awesome! Let’s get something meaningful. Suppose we want to know what OS is running on all of our nodes:

An important note: ensure that all of your puppet nodes can connect to the puppetdb server on the appropriate SSL-enabled Jetty port (by default, 8081) so they can report in.

With Puppet DB now configured, I can start writing some reports, and using exported resources.