org.apache.hadoop.hdfs.server.namenode
Class BlockPlacementPolicyDefault

java.lang.Object
  extended by org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicy
      extended by org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyDefault

@InterfaceAudience.Private
public class BlockPlacementPolicyDefault
extends BlockPlacementPolicy

The class is responsible for choosing the desired number of targets for placing block replicas. The replica placement strategy is that if the writer is on a datanode, the 1st replica is placed on the local machine, otherwise a random datanode. The 2nd replica is placed on a datanode that is on a different rack. The 3rd replica is placed on a datanode which is on a different node of the rack as the second replica.


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicy
BlockPlacementPolicy.NotEnoughReplicasException
 
Method Summary
 DatanodeDescriptor chooseReplicaToDelete(FSInodeInfo inode, Block block, short replicationFactor, Collection<DatanodeDescriptor> first, Collection<DatanodeDescriptor> second)
          Decide whether deleting the specified replica of the block still makes the block conform to the configured block placement policy.
 DatanodeDescriptor[] chooseTarget(FSInodeInfo srcInode, int numOfReplicas, DatanodeDescriptor writer, List<DatanodeDescriptor> chosenNodes, long blocksize)
          choose numOfReplicas data nodes for writer If not, return as many as we can.
 DatanodeDescriptor[] chooseTarget(String srcPath, int numOfReplicas, DatanodeDescriptor writer, List<DatanodeDescriptor> chosenNodes, HashMap<org.apache.hadoop.net.Node,org.apache.hadoop.net.Node> excludedNodes, long blocksize)
          choose numOfReplicas data nodes for writer to re-replicate a block with size blocksize If not, return as many as we can.
 DatanodeDescriptor[] chooseTarget(String srcPath, int numOfReplicas, DatanodeDescriptor writer, List<DatanodeDescriptor> chosenNodes, long blocksize)
          choose numOfReplicas data nodes for writer to re-replicate a block with size blocksize If not, return as many as we can.
 void initialize(org.apache.hadoop.conf.Configuration conf, FSClusterStats stats, org.apache.hadoop.net.NetworkTopology clusterMap)
          Used to setup a BlockPlacementPolicy object.
 int verifyBlockPlacement(String srcPath, LocatedBlock lBlk, int minRacks)
          Verify that the block is replicated on at least minRacks different racks if there is more than minRacks rack in the system.
 
Methods inherited from class org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicy
getInstance
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

initialize

public void initialize(org.apache.hadoop.conf.Configuration conf,
                       FSClusterStats stats,
                       org.apache.hadoop.net.NetworkTopology clusterMap)
Used to setup a BlockPlacementPolicy object. This should be defined by all implementations of a BlockPlacementPolicy.

Specified by:
initialize in class BlockPlacementPolicy
Parameters:
conf - the configuration object
stats - retrieve cluster status from here
clusterMap - cluster topology

chooseTarget

public DatanodeDescriptor[] chooseTarget(String srcPath,
                                         int numOfReplicas,
                                         DatanodeDescriptor writer,
                                         List<DatanodeDescriptor> chosenNodes,
                                         long blocksize)
choose numOfReplicas data nodes for writer to re-replicate a block with size blocksize If not, return as many as we can.

Parameters:
srcPath - the file to which this chooseTargets is being invoked.
numOfReplicas - additional number of replicas wanted.
writer - the writer's machine, null if not in the cluster.
chosenNodes - datanodes that have been chosen as targets.
blocksize - size of the data to be written.
Returns:
array of DatanodeDescriptor instances chosen as target and sorted as a pipeline.

chooseTarget

public DatanodeDescriptor[] chooseTarget(String srcPath,
                                         int numOfReplicas,
                                         DatanodeDescriptor writer,
                                         List<DatanodeDescriptor> chosenNodes,
                                         HashMap<org.apache.hadoop.net.Node,org.apache.hadoop.net.Node> excludedNodes,
                                         long blocksize)
choose numOfReplicas data nodes for writer to re-replicate a block with size blocksize If not, return as many as we can.

Parameters:
srcPath - the file to which this chooseTargets is being invoked.
numOfReplicas - additional number of replicas wanted.
writer - the writer's machine, null if not in the cluster.
chosenNodes - datanodes that have been chosen as targets.
blocksize - size of the data to be written.
Returns:
array of DatanodeDescriptor instances chosen as target and sorted as a pipeline.

chooseTarget

public DatanodeDescriptor[] chooseTarget(FSInodeInfo srcInode,
                                         int numOfReplicas,
                                         DatanodeDescriptor writer,
                                         List<DatanodeDescriptor> chosenNodes,
                                         long blocksize)
choose numOfReplicas data nodes for writer If not, return as many as we can. The base implemenatation extracts the pathname of the file from the specified srcInode, but this could be a costly operation depending on the file system implementation. Concrete implementations of this class should override this method to avoid this overhead.

Parameters:
srcInode - The inode of the file for which chooseTarget is being invoked.
numOfReplicas - additional number of replicas wanted.
writer - the writer's machine, null if not in the cluster.
chosenNodes - datanodes that have been chosen as targets.
blocksize - size of the data to be written.
Returns:
array of DatanodeDescriptor instances chosen as target and sorted as a pipeline.

verifyBlockPlacement

public int verifyBlockPlacement(String srcPath,
                                LocatedBlock lBlk,
                                int minRacks)
Verify that the block is replicated on at least minRacks different racks if there is more than minRacks rack in the system.

Specified by:
verifyBlockPlacement in class BlockPlacementPolicy
Parameters:
srcPath - the full pathname of the file to be verified
lBlk - block with locations
minRacks - number of racks the block should be replicated to
Returns:
the difference between the required and the actual number of racks the block is replicated to.

chooseReplicaToDelete

public DatanodeDescriptor chooseReplicaToDelete(FSInodeInfo inode,
                                                Block block,
                                                short replicationFactor,
                                                Collection<DatanodeDescriptor> first,
                                                Collection<DatanodeDescriptor> second)
Decide whether deleting the specified replica of the block still makes the block conform to the configured block placement policy.

Specified by:
chooseReplicaToDelete in class BlockPlacementPolicy
Parameters:
inode - The inode of the file to which the block-to-be-deleted belongs
block - The block to be deleted
replicationFactor - The required number of replicas for this block
first - The replica locations of this block that are present on at least two unique racks.
second - Replica locations of this block that are not listed in the previous parameter.
Returns:
the replica that is the best candidate for deletion


Copyright © 2009 The Apache Software Foundation