HDFS (Hadoop Distributed File System) is designed to store large amounts of data reliably and to stream those data to user applications at high bandwidth. One of the key features of HDFS is its fault tolerance, which ensures data availability and reliability even in the event of hardware failures.

1. Data Replication:

   • HDFS stores multiple copies of data blocks across different nodes in the cluster.
   • The default replication factor is 3, meaning each block of data is stored on three different nodes.
   • This redundancy ensures that if one node fails, the data can still be accessed from another node.

2. Heartbeat and Block Reports:

   • DataNodes send periodic heartbeats to the NameNode to confirm their availability.
   • Block reports are sent by DataNodes to the NameNode, listing all the blocks they store.
   • If a DataNode fails to send a heartbeat, the NameNode marks it as dead and initiates the replication of its blocks to other DataNodes.

3. Rack Awareness:

   • HDFS is rack-aware, meaning it understands the network topology of the cluster.
   • Data is replicated across different racks to ensure that even if an entire rack fails, the data is still available.
   • Typically, one replica is stored on a different rack than the other two.

4. Data Integrity:

   • HDFS uses checksums to verify the integrity of data blocks.
   • When a block is read, its checksum is computed and compared with the stored checksum to detect any corruption.
   • If corruption is detected, the block is re-replicated from another replica.

5. Automatic Recovery:

   • When a DataNode fails, the NameNode automatically re-replicates the blocks stored on the failed node to other healthy nodes.
   • This ensures that the replication factor is maintained and data remains available.

You can set the replication factor for a file in HDFS using the hdfs dfs command. Here’s an example:

HDFS provides a command to check the integrity of files:

To understand how HDFS handles DataNode failures, you can manually stop a DataNode and observe the behavior:

After stopping the DataNode, you can check the HDFS web UI or use the hdfs dfsadmin command to see how the NameNode handles the failure and re-replicates the blocks.

1. Objective: Simulate a DataNode failure and observe how HDFS handles the fault tolerance.
2. Steps:
   • Upload a file to HDFS.
   • Check the replication factor and ensure it is set to 3.
   • Manually stop a DataNode.
   • Observe the NameNode’s response and the re-replication process.
   • Restart the DataNode and verify the file’s availability.

1. Upload a file to HDFS:

   hdfs dfs -put localfile.txt /user/hadoop/localfile.txt
   

2. Check the replication factor:

   hdfs fsck /user/hadoop/localfile.txt -files -blocks -racks
   

3. Manually stop a DataNode:

   stop-dfs.sh
   

4. Observe the NameNode’s response:

   • Check the HDFS web UI or use the following command:

     hdfs dfsadmin -report
     

5. Restart the DataNode:

   start-dfs.sh
   

6. Verify the file’s availability:

   hdfs dfs -cat /user/hadoop/localfile.txt
   

HDFS fault tolerance is a critical feature that ensures data reliability and availability in a Hadoop cluster. By understanding and leveraging data replication, heartbeat mechanisms, rack awareness, data integrity checks, and automatic recovery, HDFS can handle hardware failures gracefully. This module has provided an overview of these concepts, practical examples, and an exercise to reinforce your understanding. In the next module, we will delve deeper into MapReduce programming and its integration with HDFS.