Introduction

The previous article covered the configuration of two resource groups, each containing a failover zpool for use as the zonepath to a highly-available zone, and a failover IP address to be assigned to each zone. The two zones were also configured and installed, and we verified that they could be booted on either node of the cluster, provided that the storage had been failed over appropriately and was available on the node where the zone was being booted.

This final part in the series will cover the incorporation of the zone boot/shutdown/failover into the cluster framework, as well as the configuration of two iPlanet resources to illustrate how Solaris Cluster can manage SMF services deployed within a highly-available Solaris zone.

Highly-Available Zones

First, install the ha-zones data service, if you haven’t done so already. I installed the full cluster package suite, so already have all data services at my disposal:

Register the SUNW.gds resource type:

This is the Generic Data Service that is utilised by SUNWsczone (HA for Solaris Containers) for deploying highly-available zones. SUNWsczone supplies three highly-available mechanisms for zone deployment - sczbt (zone boot - used to start/stop/failover zones), sczsh (zone script resource - used for deploying highly-available services within zones, with start/stop scripts to control them) and sczsmf (zone SMF resource, used for deploying highly-available services within zones, with SMF services to control them). We’ll be using both sczbt and sczsmf.

To register our highly-available failover zones as clustered resources, a template configuration file and registration script are provided, both of which are located under /opt/SUNWsczone/sczbt/util. To start configuring the resource, take a copy of sczbt_config and give it an appropriate name - one per zone. Whilst these files are not required after the resource is registered, it’s worth keeping them for future reference. To that end, I create them on both nodes.

Edit each file - the template is very well documented within its comments. For ha-zone-1:

and for ha-zone-2:

I’ll summarise the directives here below:

• RS - The name of the failover zone resource to be created
• RG - The name of the resource group containing RG
• PARAMETERDIR - Path of the parameter directory used to contain the zone boot resource parameters (this will be created shortly)
• SC_NETWORK - true if SUNW.LogicalHostname resources are used, false if the zone has its interface(s) configured via traditional zonecfg methods
• Zonename - The name of the zone
• Zonebrand - Brand of the zone. Ours are straight Solaris 11, which is brand solaris
• Milestone - SMF Milestone which needs to be online before the zone is considered booted - we specify svc:/milestone/multi-user-server
• Zonebootopt - Any zone boot options required
• LXrunlevel - Only used with the lx Zonebrand (i.e. Linux branded zones) - runlevel which needs to get reached before zone is considered booted
• SLrunlevel - Solaris legacy runlevel which needs to get reached before the zone is considered booted (only used with solaris8 and solaris9 Zonebrands)
• Mounts - A list of directories and their mount options to be loopback mounted from the global zone.

Take a look at the original sczbt_config for more details.

Next, create PARAMETERDIR as defined above. This should be on shared storage - I create them at <zonepath>/params - so do this on the node currently mastering the resources (clusternode2 in my case):

Now, the zone boot resources can be registered. Still within the /opt/SUNWsczone/sczbt/util directory, run the sczbt_register script against each configuration file:

You can now enable the resources:

On clusternode2, you should see that both zones are now in a running state:

Try ssh-ing into the zones via their respective logical hostnames:

And verify the status of the resources with clresource status:

Cool - everything is online! If you take a look at the sczbt_register script, you will see that the appropriate resource dependencies on the HAStoragePlus and LogicalHostname resources are created. This can be verified with a command such as:

As the zones are running on independent zpools, we can fail one of the resource groups over to clusternode1, and confirm correct operation:

Done - we now have two highly-available Solaris zones. But zones are no good without a service to host. Let’s configure iPlanet to run as an SMF service under ha-zone-1, and then configure the cluster to manage it.

Zone SMF Configuration and Clustering

In this section, I’ll install Oracle iPlanet 7.0.15 into ha-zone-1, and configure it be managed by the cluster. I’ll write some very basic probe scripts to provide health monitoring (these scripts can be as simple or as complex as your monitoring requirements dictate).

First, unzip the iPlanet installation media to a suitable location:

Launch ./setup, and run through the installation wizard. I accepted defaults for everything for the purposes of this demostration and installed to /opt/oracle/webserver7 - this created an administration server and a web server instance. I did, however, deviate from defaults to run 64-bit runtimes, and to create SMF services for the server instances - we need those for our clustered sczsmf resources. I could have chosen “no” to SMF service creation, and instead use sczsh resources (specify start/stop scripts instead of SMF services), but it’s cleaner to use SMF services if they are available. Here is my session:

Once the installation was complete, I logged in via wadm and confirmed that the instance had been created (and that the admin server was running):

As you can see above, my old iPlanet admin ways got the better of me, and I started the ha-zone-1 instance via wadm. No matter - it updated its status with SMF anyway. Verifying both services via svcs we can see they’re online:

Here, http:admin-server is the administration server instance, listening on port 8989 (with SSL), and http:https-ha-zone-1 is the standard web server instance, listening on port 80.

Next, create some basic probe scripts. Exit statuses should be defined as follows:

0	Successful completion
100	An error occurred that requires a resource restart
201	An error has occurred that requires an immediate failover of the resource group

I created a directory /opt/probes within the zone, and created two very simple probe scripts. Crappily simple. For Production, do much better please. For my simple tests, these are fine.

Next, navigate to /opt/SUNWsczone/sczsmf/util, and perform similar steps to registering the zone boot resources (sczbt_{register,config}). Create two copies of the sample (and again, well documented) sczsmf_config file, and update as required:

Again, I’ll summarise the configuration directives:

• RG - Name of the resource
• RG - Name of the resource group containing RS
• SCZBT_RS - Name of the SC Zone boot resource (e.g. ha-zone-1-res)
• ZONE - The name of the zone
• SERVICE - Either the full FMRI or unique pattern that matches the service
• RECURSIVE - false to just enable the service and no dependants, or true to enable the service and recursively enable dependants
• STATE - false to not wait until desired service state is reached, or true to wait
• SERVICE_PROBE - path to probe script to check the SMF service

Whilst relative paths work when running sczbt_register - they don’t with sczsmf_register - observe:

So, specify the absolute path, and register the SMF resources:

Next, enable the resources:

No errors? Good - check the resource status for the entire resource group:

All resources are online. Test by browsing to https://ha-zone-1:8989 and http://ha-zone-1. You should see the Administration Console, and the default test page, respectively.

Perform a failover to clusternode2:

And verify that you can still hit the webserver. We are done!

Final Task - Quorum Cleanup

As a final task, let’s migrate the quorum device off of the quorum server, and onto one of our shared DID devices. This will enable the quorum server to be decommissioned - making the topology simpler.

This is as simple as adding the new quorum device, removing the old quorum device, and verifying the status:

Conclusion

This final article in my Solaris Cluster 4.1 “Cluster in a box” series has shown how to bring two zones with their zonepaths on shared storage under cluster control, and how to deploy a highly-available SMF managed service to one of those zones - all managed by the cluster.

It has also briefly shown how to move the quorum device away from the quorum server and onto a shared DID device.

I’ll be writing more about Solaris Cluster as I deploy more services and delve deeper.