Version 11.4.7 (2016-09-08)
These release notes apply to all SUSE Linux Enterprise High Availability
Extension 11 Services Pack 4 based products (e.g. for x86, x86_64,
Itanium, Power and System z). Some sections may not apply to a
particular architecture or product. Where this is not obvious, the
respective architectures are listed explicitly in these notes.
Instructions for installing SUSE Linux Enterprise High Availability
Extension can be found in the
README file on the
Manuals can be found in the
docu directory of the
installation media. Any documentation (if installed) can be found in the
/usr/share/doc/ directory of the installed system.
This SUSE product includes materials licensed to SUSE under the GNU General Public License (GPL). The GPL requires SUSE to provide the source code that corresponds to the GPL-licensed material. The source code is available for download at http://www.suse.com/download-linux/source-code.html. Also, for up to three years after distribution of the SUSE product, upon request, SUSE will mail a copy of the source code. Requests should be sent by e-mail to mailto:firstname.lastname@example.org or as otherwise instructed at http://www.suse.com/download-linux/source-code.html. SUSE may charge a reasonable fee to recover distribution costs.
SUSE Linux Enterprise High Availability Extension is an affordable, integrated suite of robust open source clustering technologies that enable enterprises to implement highly available Linux clusters and eliminate single points of failure.
Used with SUSE Linux Enterprise Server, it helps firms maintain business continuity, protect data integrity, and reduce unplanned downtime for their mission-critical Linux workloads.
SUSE Linux Enterprise High Availability Extension provides all of the essential monitoring, messaging, and cluster resource management functionality of proprietary third-party solutions, but at a more affordable price, making it accessible to a wider range of enterprises.
It is optimized to work with SUSE Linux Enterprise Server, and its tight integration ensures customers have the most robust, secure, and up to date high availability solution. Based on an innovative, highly flexible policy engine, it supports a wide range of clustering scenarios.
With static or stateless content, the High Availability cluster can be used without a cluster file system. This includes web-services with static content as well as printing systems or communication systems like proxies that do not need to recover data.
Finally, its open source license minimizes the risk of vendor lock-in, and it's adherence to open standards encourages interoperability with industry standard tools and technologies.
In Service Pack 4, a number of improvements have been added, some of which are called out explicitly here. For the full list of changes and bugfixes, refer to the change logs of the RPM packages. These changes are in addition to those that have already been added with Service Pack 1, 2, and 3. Here are some highlights:
SUSE Linux Enterprise High Availability Extension 11 Service Pack 4 includes a new feature called pacemaker_remote. It allows nodes not running the cluster stack (pacemaker+corosync) to integrate into the cluster and have the cluster manage their resources just as if they were a real cluster node. This feature makes it ideal to deploy large scale SAP deployment by supporting worker nodes as scale-out clustering option. With this feature, you can deploy a HA cluster with up to 40+ nodes.
SP4 improves the usability in HAWK by introducing new templates and wizards. Oracle and DB2 related templates and wizards make SP4 easier to use to boost your database availability.
This section includes an overview of some of the major features and new functionality provided by SUSE Linux Enterprise High Availability Extension 11 SP4.
Data replication is part of a disaster prevention strategy in most large enterprises. Using network connections data is replicated between different nodes to ensure consistent data storages in case of a site failure.
Data replication is provided in SUSE Linux Enterprise High Availability Extension 11 SP4 with DRBD. This software based data replication allows customers to use different types of storage systems and communication layers without vendor lock-in. At the same time, data replication is deeply integrated into the operating system and thus provide ease-of-use. Features related to data replication and included with this product release are:
YaST setup tools to assist initial setup
Fully synchronous, memory synchronous or asynchronous modes of operation
Differential storage resynchronization after failure
Bandwidth of background resynchronization tunable
Shared secret to authenticate the peer upon connect
Configurable handler scripts for various DRBD events
Online data verification
With these features data replication can be easier configured and used. And with improved storage resynchronization recovery times will be decreased significantly.
The distributed replicated block device (DRBD) version included supports active/active mirroring, enabling the use of services such as cLVM2 or OCFS2 on top.
Linux Virtual Server (LVS) is an advanced IP load balancing solution for Linux. IP load balancing provides a high-performance, scalable network infrastructure. Such infrastructure is typically used by enterprise customers for webservers or other network related service workloads.
With LVS network requests can be spread over multiple nodes to scale the available resources and balance the resulting workload. By monitoring the compute nodes, LVS can handle node failures and redirect requests to other nodes maintaining the availability of the service.
STONITH plugin did not work as expected. This plug-in checks if a
kernel dump is in progress on a node. If so, it returns
true, and acts as if the node
has been fenced. This avoids fencing a node that is already down but
doing a dump, which takes some time.
stonith:fence_kdump resource agent (provided
by the package
fence-agents) to monitor all nodes
with the kdump function enabled. In
/etc/sysconfig/kdump, configure KDUMP_POSTSCRIPT to
send a notification to all nodes when the kdump process is finished.
The node that does a kdump will restart automatically after kdump has
Do not forget to open a port in the firewall for the
fence_kdump resource. The default port is
Setting up Linux Virtual Server (LVS) in combination with
SMTP did not work due to missing
ipvs support. IPVS (IP Virtual Server) is used for
Linux Virtual Server.
iptables package has been updated to a higher
version and now includes the match support of iptables. To make it
work, you need the probe kernel module
is provided by the
In previous Service Packs of SLE HA 11, SBD STONITH resources for two-node clusters could be configured with:
primitive stonith-sbd stonith:external/sbd \ op start start-delay=15
The start operation configured with a
double-fencing (both nodes being fenced) in case of a split-brain
situation. In SLE HA 11 SP4, this solution no longer prevents
double-fencing and must not be used anymore.
In SLE HA 11 SP4, we introduced a feature called "random fencing delay" for the same purpose.
Configure the SBD resource with the parameter
pcmk_delay_max, for example:
primitive stonith-sbd stonith:external/sbd \ params pcmk_delay_max=30
pcmk_delay_max enables random fencing
delay for the SBD fencing device and specifies a maximum number of
seconds for the random delay.
It may be desired to start and stop multiple resources all at once, without having explicit dependencies between those resources.
This feature adds resource tags to Pacemaker. Tags are collections of resources that do not imply any colocation or ordering constraints, but can be referenced in constraints or when starting and stopping resources.
Sometimes, it is desired that two resources should run on the same node. However, there should be no further dependency implied between the two resources, so that if one fails, the other one can keep running.
This can be accomplished using a third, dummy resource which both resources depend on in turn. To make it easier to create a configuration like this, the command "assist weak-bond" has been added to crmsh.
In the past you have had to start the cluster at boot unconditionally, if you want to make sure that the cluster stops when the server stops.
Now it is possible to the cluster later.
The openais service is still in sysconfig, but additionally, we now have a parameter START_ON_BOOT=Yes/No in /etc/sysconfig/openais.
If START-ON_BOOT=Yes (default), the openais service will start at boot.
If START_ON_BOOT=No, the openais service will not start at boot. Then you can start it manually whenever you want to start it.
Multiple wizards have been added to hawk to ease configuration, including cLVM + MD-RAID and DRBD.
Hawk now includes a wizard for configuring a cluster resource for the DB2 HADR database.
Hawk now includes a wizard for configuring a cluster resource for the DB2 database.
Hawk now includes a wizard for configuring cluster resources for the Oracle database.
When Relax-and-Recover version 1.10.0 does not work the newer version 1.16 could help.
In addition to Relax-and-Recover version 1.10.0 that is still provided in the RPM package "rear" we provide Relax-and-Recover version 1.16 as additional totally separated RPM package "rear116".
A new separated package name rear116 is used so that users where version 1.10.0 does not support their particular needs can manually upgrade to version 1.16 but on the other hand users who have a working disaster recovery procedure with version 1.10.0 do not need to upgrade. Therefore the package name contains the version and the packages conflict with each other to avoid that an installed version may get accidentally replaced with another version.
When you have a working disaster recovery procedure, do not change it!
For each rear version upgrade you must carefully re-validate that your particular disaster recovery procedure still works for you.
See in particular the section "Version upgrades" at https://en.opensuse.org/SDB:Disaster_Recovery.
The SUSE Linux Enterprise High Availability Extension stack supports a wide range of different cluster topologies.
Local and Metro Area (stretched) clusters are supported as part of a SUSE Linux Enterprise High Availability Extension subscription. Geographical clustering requires an additional Geo Clustering for SUSE Linux Enterprise High Availability Extension subscription.
In a local cluster environment, all nodes are connected to the same storage network and on the same network segment; redundant network interconnects are provided. Latency is below 1 millisecond, and network bandwidth is at least 1 Gigabit/s.
Cluster storage is fully symmetric on all nodes, either provided via the storage layer itself, mirrored via MD Raid1, cLVM2, or replicated via DRBD.
In a local cluster all nodes run in a single corosync domain, forming a single cluster.
In a Metro Area cluster, the network segment can be stretched to a maximum latency of 15 milliseconds between any two nodes (approximately 20 miles or 30 kilometers in physical distance), but fully symmetric and meshed network inter-connectivity is required.
Cluster storage is assumed to be fully symmetric as in local deployments.
As a stretched version of the local cluster, all nodes in a Metro Area cluster run in a single corosync domain, forming a single cluster.
A Geo scenario is primarily defined by the network topology; network latency higher than 15 milliseconds, reduced network bandwidth, and not fully interconnected subnets. In these scenarios, each site by itself must satisfy the requirements of and be configured as a local or metropolitan cluster as defined above. A maximum of three sites are then connected via Geo Clustering for SUSE Linux Enterprise High Availability Extension; for this, direct TCP connections between the sites must be possible, and typical latency should not exceed 1 second.
Storage is typically asymmetrically replicated by the storage layer, such as DRBD, MD Raid1, or vendor-specific solutions.
DLM, OCFS2, and cLVM2 are not available across site boundaries.
The LVS TCP/UDP load balancer currently only works with Direct Routing and NAT setups. IP-over-IP tunnelling forwarding to the real servers does not currently work.
To ensure data integrity, a full RAID1 resync is triggered when a device is re-added to the mirror group. This can impact performance, and it is thus advised to use multipath IO to reduce exposure to mirror loss.
Due to the need of the cluster to keep both mirrors uptodate and consistent on all nodes, a mirror failure on one node is treated as if the failure had been observed cluster-wide, evicting the mirror on all nodes. Again, multipath IO is recommended to reduce this risk.
In situations where the primary focus is on redundancy and not on
scale-out, building a storage target node (using
raid1 in a fail-over configuration or using
drbd) and reexporting via iSCSI, NFS, or CIFS could
be a viable option.
To use quotas on ocfs2 filesystem, the filesystem has to be created with appropriate quota features: 'usrquota' filesystem feature is needed for accounting quotas for individual users, 'grpquota' filesystem feature is needed for accounting of quotas for groups. These features can be also enabled later on an unmounted filesystem using tunefs.ocfs2.
For quota-tools to operate on the filesystem, you have to mount the filesystem with 'usrquota' (and/or 'grpquota') mount option.
When a filesystem has appropriate quota feature enabled, it maintains in its metadata how much space and files each user (group) uses. Since ocfs2 treats quota information as a filesystem internal metadata, there is no need to ever run quotacheck(8) program. Instead, all the needed functionality is built into fsck.ocfs2 and the filesystem driver itself.
To enable enforcement of limits imposed on each user / group, run quotaon(8) program similarly as for any other filesystem.
Commands quota(1), setquota(8), edquota(8) work as usual with ocfs2 filesystem. Commands repquota(8) and warnquota(8) do not work with ocfs2 because of a limitation in the current kernel interface.
For performance reasons each cluster node performs quota accounting locally and synchronizes this information with a common central storage once per 10 seconds (this interval is tunable by tunefs.ocfs2 using options 'usrquota-sync-interval' and 'grpquota-sync-interval'). Thus quota information need not be exact at all times and as a consequence user / group can slightly exceed their quota limit when operating on several cluster nodes in parallel.
Support requires an appropriate subscription from SUSE; for more information, see http://www.suse.com/products/server/services_support.html.
A Geo Clustering for SUSE Linux Enterprise High Availability Extension subscription is needed to receive support and maintenance to run geographical clustering scenarios, including manual and automated setups.
Support for the DRBD storage replication is independent of the cluster scenario and included as part of the SUSE Linux Enterprise High Availability Extension product and does not require the addition of a Geo Clustering for SUSE Linux Enterprise High Availability Extension subscription.
General Support Statement
The following definitions apply:
L1: Installation and problem determination - technical support designed to provide compatibility information, installation and configuration assistance, usage support, on-going maintenance and basic troubleshooting. Level 1 Support is not intended to correct product defect errors.
L2: Reproduction of problem isolation - technical support designed to duplicate customer problems, isolate problem areas and potential issues, and provide resolution for problems not resolved by Level 1 Support.
L3: Code Debugging and problem resolution - technical support designed to resolve complex problems by engaging engineering in patch provision, resolution of product defects which have been identified by Level 2 Support.
SUSE will only support the usage of original (unchanged or not recompiled) packages.
When scripting cluster configuration changes, the scripts are more robust if changes are applied with a single commit. Creating a shadow CIB to collect the changes makes this easier, but has previously required naming the shadow CIB.
crmsh now allows the creation of shadow CIBs without explicitly specifying a name. The name will be determined automatically and will not clash with any other previously created shadow CIBs.
Read the READMEs on the CDs.
Get detailed changelog information about a particular package from the RPM:
rpm --changelog -qp <FILENAME>.rpm
<FILENAME>. is the name of the RPM.
ChangeLog file in the top level of CD1
for a chronological log of all changes made to the updated packages.
Find more information in the
docu directory of CD1
of the SUSE Linux Enterprise High Availability Extension CDs. This
directory includes a PDF version of the High Availability Guide.
http://www.suse.com/documentation/sle_ha/ contains additional or updated documentation for SUSE Linux Enterprise High Availability Extension 11.
Visit http://www.suse.com/products/ for the latest product news from SUSE and http://www.suse.com/download-linux/source-code.html for additional information on the source code of SUSE Linux Enterprise products.
Copyright (c) 2016 SUSE LLC.
Thanks for using SUSE Linux Enterprise High Availability Extension in your business.
The SUSE Linux Enterprise High Availability Extension Team.