Hitachi

In-Memory Data Grid Hitachi Elastic Application Data Store


2.9.2 EADS server shutdown decided by the agreement of a specific number of EADS servers

If there is an EADS server that does not send a heartbeat within the specified timeout (the default is two seconds), the other EADS servers in the cluster check whether that EADS server is alive (using check for live servers). If the check for live servers times out, shutdown of the EADS server is decided by the agreement of a specific number of EADS servers (the default is one EADS server). After the server shutdown is decided, the corresponding EADS server is removed from the cluster.

Because the cluster is always monitored by heartbeat transmission from EADS servers, an EADS server shutdown can be detected at an early stage.

The following figure shows the flow of detecting an EADS server shutdown.

Figure 2‒25: Flow of detecting an EADS server shutdown

[Figure]

  1. If no heartbeat is sent from EADS server 3, the EADS servers in the cluster perform a check for live servers on EADS server 3.

  2. If the check for live servers times out before a heartbeat is sent from EADS server 3, the EADS servers in the cluster mutually send heartbeats with the attached information EADS server 3 is in crisis status.

  3. If a specific number of EADS servers (the default is one EADS server) agree that EADS server 3 is in crisis status, shutdown of EADS server 3 is decided.

    The EADS servers in the cluster then decide whether EADS server 3 is to be removed from the cluster. If consensus is reached by the majority in the cluster, EADS server 3 is removed from the cluster.

    The removed EADS server is forcibly isolated. An isolated EADS server no longer accepts requests from the EADS client.

Important note

The cluster can continue operating as long as the number of EADS servers that have shut down due to failures is less than the number of data copies plus the original.

For example, if the data multiplicity is 3 and there are five EADS servers and two of the EADS servers are shut down, monitoring of the cluster can continue.