Resource Manager/Failover Manager

The Resource Manager/Failover Manager is responsible for stopping and starting resources, managing resource dependencies, and for initiating fail over of Groups. It receives resource and system state information from the Resource Monitor and the Node Manager. It uses this information to make decisions about Groups. The Failover Manager is responsible for deciding which systems in the cluster should ‘own’ which Groups. When Group arbitration finishes those systems that ‘own’ individual Groups turn control of the resources within the Group over to their respective Resource Managers. When failures of resources within a Group cannot be handled by the ‘owning’ system, then the Failover Managers re-arbitrate for ownership of the Group.

If a resource fails, the Resource Manager may choose to restart the resource, or to take the resource offline along with its dependent resources. If it takes the resource offline, it will indicate to the Failover Manager that the Group should be restarted on another system in the cluster. This is called pushing a Group to another system. A cluster administrator may also manually initiate such a Group transfer. The algorithm for both situations is identical, except that resources are gracefully shutdown for a manually initiated fail over, while they are forcefully shut down in the failure case.

When an entire system in the cluster fails, its Groups must be pulled from that system to another system. This process is similar to pushing a Group, but without the shutdown phase on the failed system. The complication here is determining what Groups were running on the failed system and which system should take ownership of the various Groups. All systems capable of hosting the Groups negotiate among themselves for ownership. This negotiation is based on system capabilities, current load, application feedback or group “system preference list”. Once negotiation of the Group is complete, all members of the cluster update their data bases and thus keep track of which system owns the Groups.

When a system comes back on-line, the Failover Manager can decide to move some groups back to that system. We refer to this action as failback. Groups must have a preferred owner defined to fail back. Groups for which the new system is the “preferred” owner will be pushed from the current owner to the new system. Protection, in the form of a timing window, is included to defend against the case where a system continually crashes as soon as it tries running an important application.

The Event Processor is the electronic switchboard that propagates events to and from applications and other components within the Cluster Service. The Event processor also performs miscellaneous services such as delivering signal events to cluster aware applications and maintaining cluster objects.

The Event Processor is responsible for initializing the Cluster Service. It is the initial entry point for the Cluster Service. After initialization is complete the external state of the system is Offline. The Event Processor will then call the Node Manager to begin the process of joining or forming a cluster.
Time Services

All systems in the cluster must maintain a consistent view of time. A special resource implements the Time Service. The node on which this resource is on line is called the Time Source. There is always a Time Source in the cluster. The goal is to ensure that the cluster members have a consistent view of the time, rather than the accurate time. The administrator can influence the decision and cause a particular system, or systems to be used as the Time Source.