Real Application Clusters Software Architecture
Some of these components are supplied with the Oracle database software while others are vendor-specific.
The Operating System-Dependent Clusterware
Real Application Clusters Shared Disk Components
Real Application Clusters-Specific Daemon and Instance Processes
The Global Cache and Global Enqueue Service
The Operating System-Dependent Clusterware
Real Application Clusters processing uses operating system-dependent (OSD) clusterware to access the operating system and for cluster-related service processing such as communicating information about instance startup and shutdown. Vendors provide the OSD clusterware for UNIX operating systems, and Oracle provides the OSD clusterware for Windows NT and Windows 2000 operating systems. The OSD has the following subcomponents:
The Cluster Manager
The Node Monitor
The Interconnect
The Cluster Manager
The cluster manager (CM) oversees internode messaging that travels over the interconnect to coordinate internode operations. The cluster manager also provides a global view of the cluster and the nodes and instances that are members of it. The cluster manager also controls cluster membership.
The Node Monitor
The cluster manager includes a subset of functionality known as a node monitor. The node monitor polls the status of each resource in the cluster including the nodes, the interconnect hardware and software, and the shared disks. In the Oracle-supplied Cluster Manager for Windows, the node monitor also polls the Oracle instances.
The cluster manager informs clients and the Oracle server when the status of cluster resources change. This is because Real Application Clusters manages cluster membership by reconfiguring the cluster database when a joining instance registers with the cluster manager or when an existing instance disconnects from it.
The node monitor also serves the cluster manager by:
Providing node management interface modules
Discovering and tracking the membership states of the nodes by providing a common view of node membership across the cluster
Detecting and diagnosing changes in the states of active nodes and communicating information about those change events
The IPC or interconnect controls messaging among the nodes. Real Application Clusters also uses the IPC to transfer data blocks between instances.
The Global Services Daemon
The Global Services Daemon (GSD) runs on each node with one GSD process per node. The GSD coordinates with the cluster manager to receive requests from clients such as the DBCA, EM, and the SRVCTL utility to execute administrative job tasks such as instance startup or shutdown. The GSD is not an Oracle instance background process and is therefore not started with the Oracle instance.
Instance Processes Specific to Real Application Clusters
A Real Application Clusters database has the same processes as single-instance Oracle databases such as process monitor (PMON), database writer (DBWRn), log writer (LGWR), and so on. There are also additional Real Application Clusters-specific processes. The exact names of these processes and the trace files that they create are platform-dependent.
Global Cache Service Processes (LMSn), where n ranges from 0 to 9 depending on the amount of messaging traffic, control the flow of messages to remote instances and manage global data block access. LMSn processes also transmit block images between the buffer caches of different instances. This processing is part of the Cache Fusion feature.
The Global Enqueue Service Monitor (LMON) monitors global enqueues and resources across the cluster and performs global enqueue recovery operations. Enqueues are shared memory structures that serialize row updates.
The Global Enqueue Service Daemon (LMD) manages global enqueue and global resource access. Within each instance, the LMD process manages incoming remote resource requests.
The Lock Process (LCK) manages non-Cache Fusion resource requests such as library and row cache requests.
The Diagnosability Daemon (DIAG) captures diagnostic data about process failures within instances. The operation of this daemon is automated and it updates an alert log file to record the activity that it performs.
The Global Cache and Global Enqueue Service
The Global Cache Service (GCS) and Global Enqueue Service (GES) are integrated components of Real Application Clusters that coordinate simultaneous access to the shared database and to shared resources within the database. These services maintain consistency and data integrity. The GCS and GES on each instance, as well as the Cluster Manager, use the IPC to communicate between instances and within the cluster. This section describes the following features of the GCS and GES:
Application Transparency
Global Resource Directory with Distributed Architecture
Resource Mastering and Affinity
GCS and GES Interaction with the Cluster Manager
Application Transparency
The coordination of access to resources that is performed by the GCS and GES is transparent to applications. Applications in Real Application Clusters use the same concurrency mechanisms as in single-instance Oracle databases.
Global Resource Directory with Distributed Architecture
The GCS and GES maintain a Global Resource Directory to record information about resources. The Global Resource Directory resides in memory, is distributed throughout the cluster, and is available to all active instances. In this distributed architecture, each node participates in the management of information in the directory. This distributed scheme provides fault tolerance and enhanced runtime performance.
The GCS and GES ensure the integrity of the Global Resource Directory even if multiple nodes fail. The shared database is always accessible if at least one instance is active after recovery is completed. The fault tolerance of the resource directory also enables Real Application Clusters instances to start and stop at any time, in any order.
Resource Mastering and Affinity
The Global Cache and Global Enqueue Services maintain information about each resource within a cluster. The GCS and GES nominate one instance to manage all information about a particular resource. This instance is called the resource master. The GCS evaluates resource mastering periodically and changes the resource master based on data access patterns. This reduces network traffic as well as resource acquisition time.
GCS and GES Interaction with the Cluster Manager
The GCS and GES operate independently of the cluster manager. However, these services rely on the cluster manager for timely and correct information about the statuses of the instances in the cluster. If these services cannot obtain the information they need from a particular instance, then Oracle shuts down the unresponsive instance. This ensures the integrity of Real Application Clusters databases because each instance must be aware of all other active instances to coordinate shared disk access.
Reference: Oracle Documentation
