hawk's cluster simulator allowed the exploration of the cluster's reaction to status events such as node down or resource monitoring failures only. Examining the predicted result for changes to the cluster configuration was only possible via the crm shell.

hawk now supports the creation of a shadow configuration, and can be asked to explore the effects of applying this new configuration to the cluster.

Pacemaker executes an initial monitor for each service on every node ("probe"). The timeout for this monitor operation could be specified by configuring a monitor operation with interval set to 0, or would otherwise be inherited from the cluster-wide default operation timeout.

To ease the need to configure a dedicated monitor timeout for the initial probe, pacemaker will now use the timeout from the most frequent configured monitor automatically. If no monitor operation is configured at all, the cluster-wide default continues to apply.

Users which relied on the probe inheriting the timeout from the cluster-wide timeout in the presence of other monitor operations are advised to configure an explicit monitor operation instead.

Pacemaker supports utilization-based resource placement in the cluster. Previously, utilization from individual resources was not summarized to the whole group. This could lead to scenarios where only a few resources in the group could be placed, but the rest would remain stopped.

The utilization configured is now summed up for the whole group so that the whole group will be placed correctly.

Some scenarios experience temporary outages of the SBD partitions. If the majority of fencing devices were lost, this would cause a node fence, even if the cluster was otherwise healthy.

SBD now takes into account if the cluster partition remains quorate and whether the local node is healthy from pacemaker's state. This functionality must be explicitly enabled by adding "-P" to /etc/sysconfig/sbd. Please review the updated sbd manpage for a full discussion of this functionality.

Cluster file systems are used to provide scalable, high performance, and highly available file access across multiple instances of SUSE Linux Enterprise High Availability Extension servers. Oracle Cluster File System 2 (OCFS2) is a POSIX-compliant shared-disk cluster file system for Linux. OCFS2 is developed under a GPL open source license.

New features included in OCFS2 with this product release are:

Beyond this, OCFS2 continues to deliver the functionality provided in the previous releases:

With these features OCFS2 can be used as generic file system for common use without previous limitations to specific workloads. Workloads for OCFS2 included in this product are, but are not limited to:

The full functionality of OCFS2 is only available in combination with the OpenAIS, corosync, and Pacemaker-based cluster stack.

The Clustered Volume Manager allows multiple nodes to read and write volumes on a single storage device at the block layer level. It features creation and reallocation of volumes on a shared storage infrastructure like SAN or iSCSI, and allows moving volumes to a different storage device during operation. It can be used for volume snapshots for later recovery if needed.

cLVM2 has been updated to the latest upstream version.

Pacemaker orchestrates the cluster's response to change events such as node failures, resource monitoring failures, permanent or transient administrative changes, and ensures that service availability is recovered.

New features introduced by Pacemaker and included with this product release are:

With unified command line support system setup, managing and integration is made easier. To extend High Availability to all types of applications, resource agent templates and templates for configuration examples are provided for customization.

The Corosync Cluster Engine is an OSI certified implementation of a complete cluster engine. This component provides membership, ordered messaging with virtual synchrony guarantees, closed process communication groups, and an extensible framework.

In response to customer demand, the Python-based GUI component (formerly packaged as pacemaker-pygui) has been split into a server and client package, allowing server installs without client software.

The new packages are called pacemaker-mgmt for the server, and pacemaker-mgmt-client for the client.

After an update from GA, the client package may not automatically be installed, depending on installer settings, and thus the hb_gui and crm_gui commands unavailable.

The resolution is to install the package manually.

New functionality provided via new packages is not automatically installed by an update, which strives to preserve the existing functionality. It is recommended to install the HA pattern manually to take advantage of all new functionality in SUSE Linux Enterprise High Availability Extension 11 SP3.

The ssh, external/ssh, and null STONITH agents have been moved to the libglue-devel package, and are no longer installed by default.

These fencing agents are not suitable for production environments and should only be used for limited functionality demo setups. This move clarifies their intended use case.

The legacy O2CB in-kernel stack of OCFS2 is only supported in combination with Oracle RAC. Oracle RAC, due to its technical limitations, cannot be combined with the pacemaker-based cluster stack.

SUSE Linux Enterprise High Availability Extension 11 SP3 includes the Samba CTDB extension, including an OCF-compliant resource agent to orchestrate fail-over. This is fully supported, together with exporting Samba CTDB from OCFS2.

Due to technical limitations, this also includes the CTDB internal fail-over functionality for IP address take-over. Note that this part is not supported by SUSE. Only Pacemaker clusters are fully supported.

The smb_private_dir parameter for the CTDB resource agent is now deprecated and has been made optional. Existing installations using CTDB should remove this parameter from their configuration at their next convenience.

Several new parameters have been added to the CTDB resource agent in this release—run "crm ra info CTDB" for details. Two of these parameters, ctdb_manages_samba and ctdb_manages_winbind, default to "yes" for compatibility with the previous releases. Existing installations should update their configuration to explicitly set these parameters to "yes", as the defaults will be changed to "no" in a future release.

The new version of DRBD included in SUSE Linux Enterprise High Availability Extension 11 SP3 also supplies a new, updated Open Clustering Framework resource agent from the provider linbit.

It is recommended that setups are converted from ocf:heartbeat:drbd to use the new ocf:linbit:drbd agent. Some new features, such as dual-primary support for master resources, is only available in the new version.

Whereas SUSE Linux Enterprise Server 10 clusters utilized heartbeat as the cluster infrastructure layer, providing messaging and membership services, SUSE Linux Enterprise 11 High-Availability Extension uses corosync and openais. heartbeat is no longer included with the product.

Please use the hb2openais.sh tool for migrating your SUSE Linux Enterprise Server 10 environment to SUSE Linux Enterprise High Availability Extension 11 SP3.

Since EVMS2 has been depreciated in SUSE Linux Enterprise Server 11, the clustered extensions are also no longer available in SUSE Linux Enterprise High Availability Extension 11 SP3. A conversion tool is supplied as part of the lvm2-clvm package. After the conversion, the former C-EVMS2 segments can be used as regular, full-featured LVM2 logical volumes.

For more details, refer to /usr/share/doc/packages/lvm2-clvm/README.csm-converter.

The group command within the configuration submenue has been extended. The group command is now able to extend existing groups by adding new resources. Resources can also be deleted.

The group command within the configuration submenue has been extended. The group command is now able to extend existing groups by adding new resources. Resources can also be deleted.

In a local cluster environment, all nodes are connected to the same storage network and on the same network segment; redundant network interconnects are provided. Latency is below 1 millisecond, and network bandwidth is at least 1 Gigabit/s.

Cluster storage is fully symmetric on all nodes, either provided via the storage layer itself, mirrored via MD Raid1, cLVM2, or replicated via DRBD.

In a local cluster all nodes run in a single corosync domain, forming a single cluster.

In a Metro Area cluster, the network segment can be stretched to a maximum latency of 15 milliseconds between any two nodes (approximately 20 miles or 30 kilometers in physical distance), but fully symmetric and meshed network inter-connectivity is required.

Cluster storage is assumed to be fully symmetric as in local deployments.

As a stretched version of the local cluster, all nodes in a Metro Area cluster run in a single corosync domain, forming a single cluster.

A Geo scenario is primarily defined by the network topology; network latency higher than 15 milliseconds, reduced network bandwidth, and not fully interconnected subnets. In these scenarios, each site by itself must satisfy the requirements of and be configured as a local or metropolitan cluster as defined above. A maximum of three sites are then connected via Geo Clustering for SUSE Linux Enterprise High Availability Extension; for this, direct TCP connections between the sites must be possible, and typical latency should not exceed 1 second.

Storage is typically asymmetrically replicated by the storage layer, such as DRBD, MD Raid1, or vendor-specific solutions.

DLM, OCFS2, and cLVM2 are not available across site boundaries.

The LVS TCP/UDP load balancer currently only works with Direct Routing and NAT setups. IP-over-IP tunnelling forwarding to the real servers does not currently work.

The CTDB resource should be stopped on all nodes prior to update. Rolling CTDB updates are not supported for this release, due to the risk of corruption on nodes running previous CTDB versions.

To ensure data integrity, a full RAID1 resync is triggered when a device is re-added to the mirror group. This can impact performance, and it is thus advised to use multipath IO to reduce exposure to mirror loss.

Due to the need of the cluster to keep both mirrors uptodate and consistent on all nodes, a mirror failure on one node is treated as if the failure had been observed cluster-wide, evicting the mirror on all nodes. Again, multipath IO is recommended to reduce this risk.

In situations where the primary focus is on redundancy and not on scale-out, building a storage target node (using md raid1 in a fail-over configuration or using drbd) and reexporting via iSCSI, NFS, or CIFS could be a viable option.

To use quotas on ocfs2 filesystem, the filesystem has to be created with appropriate quota features: 'usrquota' filesystem feature is needed for accounting quotas for individual users, 'grpquota' filesystem feature is needed for accounting of quotas for groups. These features can be also enabled later on an unmounted filesystem using tunefs.ocfs2.

For quota-tools to operate on the filesystem, you have to mount the filesystem with 'usrquota' (and/or 'grpquota') mount option.

When a filesystem has appropriate quota feature enabled, it maintains in its metadata how much space and files each user (group) uses. Since ocfs2 treats quota information as a filesystem internal metadata, there is no need to ever run quotacheck(8) program. Instead, all the needed functionality is built into fsck.ocfs2 and the filesystem driver itself.

To enable enforcement of limits imposed on each user / group, run quotaon(8) program similarly as for any other filesystem.

Commands quota(1), setquota(8), edquota(8) work as usual with ocfs2 filesystem. Commands repquota(8) and warnquota(8) do not work with ocfs2 because of a limitation in the current kernel interface.

For performance reasons each cluster node performs quota accounting locally and synchronizes this information with a common central storage once per 10 seconds (this interval is tunable by tunefs.ocfs2 using options 'usrquota-sync-interval' and 'grpquota-sync-interval'). Thus quota information need not be exact at all times and as a consequence user / group can slightly exceed their quota limit when operating on several cluster nodes in parallel.

SLE HA customers desire a long-term statement about support and active development for the cluster stack's graphical user interface. The supported GUI should be flexible, usable without client-side installation of additional packages, firewall-friendly and also be usable from non-Linux desktops as well as mobile devices.

SUSE is committed to providing an easy to use yet powerful GUI for SLE HA. Because it best meets customer requirements, effort is being focused on the web-based frontend (hawk); new features and functionality will primarily be developed and improved in this user interface. hawk already provides the best GUI experience on SLE HA today.

The X11-based "hb_gui", which required a Linux client or an X11 server, is now in maintenance mode and is not scheduled to receive new functionality during the SLE HA 11 lifecycle, and will no longer be part of SLE HA 12.

The "ocf:heartbeat:oracle" resource agent has been improved to no longer trigger audit logs to be written, reducing the logging overhead of the cluster.