Hadoop-Let us Admin

Failure Types

The primary objective of the HDFS is to store data reliably even in the presence of failures. The three common types of failures are

  • NameNode failures
  • DataNodes failures
  • network partitions

Several things can cause loss of connectivity between name node and data nodes. Therefore, each data node is expected to send a periodic heartbeat messages to its name node. This is required to detect loss of connectivity if it stops receiving them. The name node marks data nodes as dead data nodes if they are not responding to heartbeats and refrains from sending further requests to them. Data stored on a dead node is no longer available to an HDFS client from that node, which is effectively removed from the system.