We all know the pain of configuring probe-based IPMP under Solaris, with a slew of test addresses being required, and a long line of ifconfig configuration in our /etc/hostname.<interface> files.
With Solaris 11, there is a new type of probe-based IPMP called transitive probing. This new type of probing does not require test addresses, as per the documentation: “Transitive probes are sent by the alternate interfaces in the group to probe the active interface. An alternate interface is an underlying interface that does not actively receive any inbound IP packets”.
In this article, I will configure failover (active/passive) IPMP on clusternode1 (the first node of a Solaris Cluster I’m building). Interface net0 has an address of 10.1.1.80 (configured at install time), and I’ll be adding this into an IPMP group ipmp0 along with a standby interface, net1. Make sure you are performing these steps via a console connection, as the original address associated with net0 will need to be removed before attempting to add it to an IPMP group.
The first step, ensure that there is an entry in /etc/hosts for the IP address you’re configuring IPMP for:
|
1 2 |
# grep '^10\.1\.1\.80' /etc/hosts 10.1.1.80 clusternode1 |
Next, ensure that automatic network configuration is disabled. In my case it was as I’d configured networking manually during the installation of Solaris 11:
|
1 2 3 4 |
# netadm list -p ncp -x TYPE PROFILE STATE AUXILIARY STATE ncp Automatic disabled disabled by administrator ncp DefaultFixed online active |
Verify that the appropriate physical interfaces are available. In the following output, I’ll be bonding e1000g0 (net0) and e1000g1 (net1) into a failover IPMP group.
|
1 2 3 4 5 6 |
# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net1 Ethernet unknown 0 unknown e1000g1 net2 Ethernet unknown 0 unknown e1000g2 net3 Ethernet unknown 0 unknown e1000g3 net0 Ethernet up 1000 full e1000g0 |
List the current addresses - from the output of ipadm show-addr I can see that I’ll need to delete net0/v4 and net0/v6, otherwise I’ll be unable to add net0 to the IPMP group.
|
1 2 |
# ipadm delete-addr net0/v4 # ipadm delete-addr net0/v6 |
As the net0 IP interface is already created, I only need to create the net1 interface:
|
1 |
# ipadm create-ip net1 |
I can then create the IPMP group, which I’ll call ipmp0:
|
1 |
# ipadm add-ipmp -i net0 -i net1 ipmp0 |
Next, enable transitive probing, which is disabled by default:
|
1 2 3 4 |
# svccfg -s svc:/network/ipmp setprop config/transitive-probing=true # svccfg -s svc:/network/ipmp listprop config/transitive-probing config/transitive-probing boolean true # svcadm refresh svc:/network/ipmp:default |
And configure the appropriate interface (in my case net1) to be a standby interface (as I’m using failover):
|
1 |
# ipadm set-ifprop -p standby=on -m ip net1 |
Now I can create my IPv4 address on the IPMP group:
|
1 2 3 4 |
# ipadm create-addr -T static -a clusternode1/24 ipmp0/v4 # ipadm show-addr ipmp0 ADDROBJ TYPE STATE ADDR ipmp0/v4 static ok 10.1.1.80/24 |
Finally, fix the default route. I removed the existing route and added a new default route using the new and correct interface - ipmp0:
|
1 2 3 4 5 6 7 8 9 10 |
# route -p delete default 10.1.1.1 # route -p add default 10.1.1.1 -ifp ipmp0 # netstat -rn -f inet Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ---------- --------- default 10.1.1.1 UG 1 0 ipmp0 10.1.1.0 10.1.1.80 U 8 388 ipmp0 127.0.0.1 127.0.0.1 UH 2 554 lo0 |
You can use ipmpstat to verify the configuration and health of the IPMP group:
|
1 2 3 4 5 6 7 8 9 10 11 |
# ipmpstat -g GROUP GROUPNAME STATE FDT INTERFACES ipmp0 ipmp0 ok 10.00s net0 (net1) # ipmpstat -a ADDRESS STATE GROUP INBOUND OUTBOUND :: down ipmp0 -- -- clusternode1 up ipmp0 net0 net0 # ipmpstat -t INTERFACE MODE TESTADDR TARGETS net1 transitive <net1> <net0> net0 routes clusternode1 10.1.1.1 |
Let’s perform a failover test. I’ll disable net0 and ensure that the clusternode1 address fails over:
|
1 2 3 4 |
# ipadm disable-if -t net0 # ipmpstat -t INTERFACE MODE TESTADDR TARGETS net1 routes clusternode1 10.1.1.1 |
It works! (and my SSH connection is still active…) - net1 is now active with the correct IP address. Let’s fail it back:
|
1 2 3 4 5 |
# ipadm enable-if -t net0 # ipmpstat -t INTERFACE MODE TESTADDR TARGETS net0 routes clusternode1 10.1.1.1 net1 transitive <net1> <net0> |
The address has failed back to net0, and again my SSH connection is still active. I can now continue with clusternode2, and the rest of the cluster install.