Introduction
During the process of configuring a CentOS KVM guest (hostname: mercury), I wanted to add a separate disk device for the /var/www filesystem so that I could configure the guest as a kickstart server with ample storage available.
To enable this, I wanted to do two things. First, add a new disk device to the virtualised guest for configuration as a new physical volume under LVM on the guest - this would be used to host /var/www. Second, I wanted to mount the CentOS (and possibly other) ISO images to the guest’s DVD drive so that I could populate the installation source tree.
Adding a Virtualised Hard Disk
Let’s start with adding the new hard disk device. dd is the command of choice here. You can create sparse files for test purposes (i.e. the full disk allocation is not consumed upon creation), but Red Hat warns against this as there can be data integrity issues, and obvious performance issues as the space is actually allocated whilst the device is being used. Therefore, the best choice is to pre-allocate the storage. This will obviously consume the full size of the device being created, but will be robust and will perform optimally.
I’ll create a 20GB volume for /var/www, under the default location for libvirt disk images - /var/lib/libvirt/images
|
1 |
# dd if=/dev/zero of=/var/lib/libvirt/images/mercury-dev-vda.img bs=1M count=20480 |
As you can see, I have created the image file with a meaningful name (in this case, <hostname>-<device-associated-on-guest>). This will make managing the disk images easier as more guests are created. This will take some time - go and make a cup of tea.
Once done, dd should produce output along the following lines:
|
1 2 3 |
20480+0 records in 20480+0 records out 21474836480 bytes (21 GB) copied, 168.871 s, 127 MB/s |
If your Linux distro comes with fallocate, you can do this in a second or too instead (depending on filesystem support - I use ext4 which does), rather than waiting on dd:
|
1 |
# fallocate -l 20G /var/lib/libvirt/images/mercury-dev-vda.img |
It’s worth noting that the original image created at guest installation (i.e. the one containing the OS) is available as /dev/sda (despite it being referenced as an IDE device in the configuration with a target of /dev/hda). So, the actual guest configuration shows an IDE drive:
|
1 2 3 |
# virsh dumpxml mercury | grep -A1 'mercury\.img' <source file='/var/lib/libvirt/images/mercury.img'/> <target dev='hda' bus='ide'/> |
However, within the guest, it’s detected as a SCSI drive at /dev/sda:
|
1 2 |
# lsscsi | grep disk [0:0:0:0] disk ATA QEMU HARDDISK 0.12 /dev/sda |
I’ll use virtio for any additional drives added, starting at /dev/vda. I could have added a SCSI HBA (via the virtio-scsi controller model) but there is no need to do this (and over-complicate things) for my requirements.
Next, fix permissions and ownership:
|
1 2 |
# chown qemu:qemu /var/lib/libvirt/images/mercury-dev-vda.img # chmod 600 /var/lib/libvirt/images/mercury-dev-vda.img |
and the SELinux context (here, I reference the original disk image, mercury.img):
|
1 |
# chcon --reference=/var/lib/libvirt/images/mercury.img /var/lib/libvirt/images/mercury-dev-vda.img |
Check that your new image appears sane (compared to the original disk image):
|
1 2 3 |
# ls -Z /var/lib/libvirt/images/mercury*.img -rw-------. qemu qemu system_u:object_r:svirt_image_t:s0:c166,c933 /var/lib/libvirt/images/mercury-dev-vda.img -rw-------. qemu qemu system_u:object_r:svirt_image_t:s0:c166,c933 /var/lib/libvirt/images/mercury.img |
OK - we’re good to move on to the next step. We need to write a basic XML file describing the disk device. Fire up your favourite editor, and create it:
|
1 2 3 4 5 6 |
# vi /var/tmp/mercury-dev-vda.xml <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='none'/> <source file='/var/lib/libvirt/images/mercury-dev-vda.img'/> <target dev='vda'/> </disk> |
Save the file, and use the virsh attach-device command to attach the new device to the guest:
|
1 2 |
# virsh attach-device --config mercury /var/tmp/mercury-dev-vda.xml Device attached successfully |
Note the use of the --config option above - ensuring that the changes persist. The manual page notes that this should affect the “next boot” of the domain, although as you can see below, the device was detected immediately without rebooting the guest.
On the guest, the new drive will be immediately detected, and available for use:
|
1 2 3 4 5 6 7 8 |
# fdisk -l /dev/vda Disk /dev/vda: 21.5 GB, 21474836480 bytes 16 heads, 63 sectors/track, 41610 cylinders Units = cylinders of 1008 * 512 = 516096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 |
The new image is also committed to the guest configuration, as seen by dumping the guest’s XML:
|
1 2 3 |
# virsh dumpxml mercury | grep vda <source file='/var/lib/libvirt/images/mercury-dev-vda.img'/> <target dev='vda' bus='virtio'/> |
The first step within the guest is to create a single partition for use as a LVM physical volume:
|
1 2 3 |
# sfdisk /dev/vda <<_EOF_ > 0,,8e > _EOF_ |
Next, create the physical volume:
|
1 2 |
# pvcreate /dev/vda1 Physical volume "/dev/vda1" successfully created |
I could extend the existing volume group across this new PV, but would rather keep it separate (to try to distinguish between the OS VG and a “data” VG), so create a new volume group:
|
1 2 3 4 5 6 |
# vgcreate vg_data /dev/vda1 Volume group "vg_data" successfully created # vgs VG #PV #LV #SN Attr VSize VFree vg_data 1 0 0 wz--n- 20.00g 20.00g vg_mercury 1 7 0 wz--n- 19.51g 1.51g |
You can see from the vgs output above that the new volume group has been created successfully of the appropriate size - 20GB.
Next, create the logical volume for /var/www - which is the entire point of the first part of this exercise. I created it with a capacity of 15GB - it will be trivial to resize and extend later if required.
|
1 2 |
# lvcreate -L15G -n lv_var_www vg_data Logical volume "lv_var_www" created |
Create an ext4 filesystem:
|
1 |
# mkfs.ext4 /dev/vg_data/lv_var_www |
I already had httpd running and serving files from /var/www, so prior to doing anything further, stopped it:
|
1 2 |
# service httpd stop Stopping httpd: [ OK ] |
Next, move the “old” /var/www sideways:
|
1 |
# mv /var/www /var/www.old |
Create an entry in /etc/fstab for the new filesystem:
|
1 2 3 |
# vi /etc/fstab ... /dev/vg_data/lv_var_www /var/www ext4 defaults 1 2 |
Create the mountpoint, and mount the new filesystem:
|
1 2 3 4 5 6 |
# mkdir /var/www # mount /var/www # df -hT /var/www Filesystem Type Size Used Avail Use% Mounted on /dev/mapper/vg_data-lv_var_www ext4 15G 166M 14G 2% /var/www |
Copy any content from the old /var/www directory to the new filesystem:
|
1 |
# cp -aR /var/www.old/. /var/www |
And ensure that the appropriate SELinux context is applied:
|
1 2 3 |
# chcon --reference=/var/www.old -R /var/www # ls -dZ /var/www drwxr-xr-x. root root system_u:object_r:httpd_sys_content_t:s0 /var/www |
If all appears well, start httpd and ensure that files are being served correctly:
|
1 2 3 4 |
# service httpd start Starting httpd: [ OK ] # curl http://localhost test |
The original /var/www directory can now be removed.
|
1 |
# rm -rf /var/www.old |
Mounting ISO Images to the Virtualised CDROM Drive
The next step is to mount the CentOS ISO images, available on the virtualisation host, to the virtualised DVD drive on the guest so that their contents can be transferred - as I’m setting up a kickstart server. Back on the virtualisation host, attach the first ISO using virsh attach-disk:
|
1 2 |
# virsh attach-disk mercury /data/iso/centos/x86_64/6.4/CentOS-6.4-x86_64-bin-DVD1.iso hdc --type cdrom --cache none --mode readonly Disk attached successfully |
I specify IDE device hdc here, which actually corresponds to SCSI CDROM device /dev/sr0 on the guest. Mount the drive within the guest:
|
1 |
# mount -t iso9660 -o ro,loop /dev/sr0 /mnt |
Ensure that the contents are as expected:
|
1 2 3 4 5 6 |
# ls /mnt CentOS_BuildTag isolinux RPM-GPG-KEY-CentOS-Debug-6 EFI Packages RPM-GPG-KEY-CentOS-Security-6 EULA RELEASE-NOTES-en-US.html RPM-GPG-KEY-CentOS-Testing-6 GPL repodata TRANS.TBL images RPM-GPG-KEY-CentOS-6 |
Perfect. Transfer the contents over to an appropriate location under /var/www for your installation tree:
|
1 2 |
# mkdir -p /var/www/virtualhosts/kickstart.local/htdocs/dists/centos/x86_64/6.4 # cp -aR /mnt/. /var/www/virtualhosts/kickstart.local/htdocs/dists/centos/x86_64/6.4 |
Once done, unmount the device:
|
1 |
# umount /mnt |
Back on the virtualisation host, attach the second ISO:
|
1 2 |
# virsh attach-disk mercury /data/iso/centos/x86_64/6.4/CentOS-6.4-x86_64-bin-DVD2.iso hdc --type cdrom --cache none --mode readonly Disk attached successfully |
And repeat the copying on the guest:
|
1 2 3 |
# mount -t iso9660 -o ro,loop /dev/sr0 /mnt # cp -aR /mnt/. /var/www/virtualhosts/kickstart.local/htdocs/dists/centos/x86_64/6.4 # umount /mnt |
Finish by applying the appropriate SELinux context across the copied files:
|
1 |
# chcon --reference=/var/www -R /var/www/virtualhosts/kickstart.local/htdocs/dists/centos/x86_64/6.4 |
The final step (aside from configuring the virtual host and kickstart, which are beyond the scope of this article) is to unmount the ISO, by providing an empty string as the source file to attach via virsh attach-disk:
|
1 2 |
# virsh attach-disk mercury "" hdc --type cdrom --cache none --mode readonly Disk attached successfully |
If you use virsh dumpxml to view under-the-hood, you can see that this removes any source file directive such as
|
1 |
<source file='/data/iso/centos/x86_64/6.4/CentOS-6.4-x86_64-bin-DVD2.iso'/> |
from the declaration for the CDROM device, and adds tray='open' to the target directive for the same device:
|
1 |
<target dev='hdc' bus='ide' tray='open'/> |
Conclusion
The more I play with KVM-based virtualisation on my CentOS servers, the more I am learning about its power and flexibility. There will be many more articles covering its features and pitfalls in the coming months. Now - on to that virtual host configuration …