Adding Storage Devices to KVM-Based Guests

Introduction

During the process of configuring a CentOS KVM guest (hostname: mercury), I wanted to add a separate disk device for the /var/www filesystem so that I could configure the guest as a kickstart server with ample storage available.

To enable this, I wanted to do two things. First, add a new disk device to the virtualised guest for configuration as a new physical volume under LVM on the guest - this would be used to host /var/www. Second, I wanted to mount the CentOS (and possibly other) ISO images to the guest’s DVD drive so that I could populate the installation source tree.

Adding a Virtualised Hard Disk

Let’s start with adding the new hard disk device. dd is the command of choice here. You can create sparse files for test purposes (i.e. the full disk allocation is not consumed upon creation), but Red Hat warns against this as there can be data integrity issues, and obvious performance issues as the space is actually allocated whilst the device is being used. Therefore, the best choice is to pre-allocate the storage. This will obviously consume the full size of the device being created, but will be robust and will perform optimally.

I’ll create a 20GB volume for /var/www, under the default location for libvirt disk images - /var/lib/libvirt/images

As you can see, I have created the image file with a meaningful name (in this case, <hostname>-<device-associated-on-guest>). This will make managing the disk images easier as more guests are created. This will take some time - go and make a cup of tea.

Once done, dd should produce output along the following lines:

If your Linux distro comes with fallocate, you can do this in a second or too instead (depending on filesystem support - I use ext4 which does), rather than waiting on dd:

It’s worth noting that the original image created at guest installation (i.e. the one containing the OS) is available as /dev/sda (despite it being referenced as an IDE device in the configuration with a target of /dev/hda). So, the actual guest configuration shows an IDE drive:

However, within the guest, it’s detected as a SCSI drive at /dev/sda:

I’ll use virtio for any additional drives added, starting at /dev/vda. I could have added a SCSI HBA (via the virtio-scsi controller model) but there is no need to do this (and over-complicate things) for my requirements.

Next, fix permissions and ownership:

and the SELinux context (here, I reference the original disk image, mercury.img):

Check that your new image appears sane (compared to the original disk image):

OK - we’re good to move on to the next step. We need to write a basic XML file describing the disk device. Fire up your favourite editor, and create it:

Save the file, and use the virsh attach-device command to attach the new device to the guest:

Note the use of the --config option above - ensuring that the changes persist. The manual page notes that this should affect the “next boot” of the domain, although as you can see below, the device was detected immediately without rebooting the guest.

On the guest, the new drive will be immediately detected, and available for use:

The new image is also committed to the guest configuration, as seen by dumping the guest’s XML:

The first step within the guest is to create a single partition for use as a LVM physical volume:

Next, create the physical volume:

I could extend the existing volume group across this new PV, but would rather keep it separate (to try to distinguish between the OS VG and a “data” VG), so create a new volume group:

You can see from the vgs output above that the new volume group has been created successfully of the appropriate size - 20GB.

Next, create the logical volume for /var/www - which is the entire point of the first part of this exercise. I created it with a capacity of 15GB - it will be trivial to resize and extend later if required.

Create an ext4 filesystem:

I already had httpd running and serving files from /var/www, so prior to doing anything further, stopped it:

Next, move the “old” /var/www sideways:

Create an entry in /etc/fstab for the new filesystem:

Create the mountpoint, and mount the new filesystem:

Copy any content from the old /var/www directory to the new filesystem:

And ensure that the appropriate SELinux context is applied:

If all appears well, start httpd and ensure that files are being served correctly:

The original /var/www directory can now be removed.

Mounting ISO Images to the Virtualised CDROM Drive

The next step is to mount the CentOS ISO images, available on the virtualisation host, to the virtualised DVD drive on the guest so that their contents can be transferred - as I’m setting up a kickstart server. Back on the virtualisation host, attach the first ISO using virsh attach-disk:

I specify IDE device hdc here, which actually corresponds to SCSI CDROM device /dev/sr0 on the guest. Mount the drive within the guest:

Ensure that the contents are as expected:

Perfect. Transfer the contents over to an appropriate location under /var/www for your installation tree:

Once done, unmount the device:

Back on the virtualisation host, attach the second ISO:

And repeat the copying on the guest:

Finish by applying the appropriate SELinux context across the copied files:

The final step (aside from configuring the virtual host and kickstart, which are beyond the scope of this article) is to unmount the ISO, by providing an empty string as the source file to attach via virsh attach-disk:

If you use virsh dumpxml to view under-the-hood, you can see that this removes any source file directive such as

from the declaration for the CDROM device, and adds tray='open' to the target directive for the same device:

Conclusion

The more I play with KVM-based virtualisation on my CentOS servers, the more I am learning about its power and flexibility. There will be many more articles covering its features and pitfalls in the coming months. Now - on to that virtual host configuration …