org.apache.hadoop.hdfs.server.datanode
Interface FSDatasetInterface

All Superinterfaces:
FSDatasetMBean
All Known Implementing Classes:
FSDataset

@InterfaceAudience.Private
public interface FSDatasetInterface
extends FSDatasetMBean

This is an interface for the underlying storage that stores blocks for a data node. Examples are the FSDataset (which stores blocks on dirs) and SimulatedFSDataset (which simulates data).


Nested Class Summary
static class FSDatasetInterface.BlockInputStreams
          This class contains the input streams for the data and checksum of a block
static class FSDatasetInterface.BlockWriteStreams
          This class contains the output streams for the data and checksum of a block
static class FSDatasetInterface.MetaDataInputStream
          This class provides the input stream and length of the metadata of a block
 
Method Summary
 void adjustCrcChannelPosition(Block b, FSDatasetInterface.BlockWriteStreams stream, int checksumSize)
          Sets the file pointer of the checksum stream so that the last checksum will be overwritten
 org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface append(Block b, long newGS, long expectedBlockLen)
          Append to a finalized replica and returns the meta info of the replica
 void checkDataDir()
          Check if all the data directories are healthy
 org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface createRbw(Block b)
          Creates a RBW replica and returns the meta info of the replica
 org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface createTemporary(Block b)
          Creates a temporary replica and returns the meta information of the replica
 void finalizeBlock(Block b)
          Finalizes the block previously opened for writing using writeToBlock.
 InputStream getBlockInputStream(Block b)
          Returns an input stream to read the contents of the specified block
 InputStream getBlockInputStream(Block b, long seekOffset)
          Returns an input stream at specified offset of the specified block
 BlockListAsLongs getBlockReport()
          Returns the block report - the full list of blocks stored
 long getLength(Block b)
          Returns the specified block's on-disk length (excluding metadata)
 FSDatasetInterface.MetaDataInputStream getMetaDataInputStream(Block b)
          Returns metaData of block b as an input stream (and its length)
 long getMetaDataLength(Block b)
          Returns the length of the metadata file of the specified block
 Replica getReplica(long blockId)
          Deprecated. 
 long getReplicaVisibleLength(Block block)
          Get visible length of the specified replica.
 Block getStoredBlock(long blkid)
           
 FSDatasetInterface.BlockInputStreams getTmpInputStreams(Block b, long blkoff, long ckoff)
          Returns an input stream at specified offset of the specified block The block is still in the tmp directory and is not finalized
 boolean hasEnoughResource()
          checks how many valid storage volumes are there in the DataNode
 ReplicaRecoveryInfo initReplicaRecovery(BlockRecoveryCommand.RecoveringBlock rBlock)
          Initialize a replica recovery.
 void invalidate(Block[] invalidBlks)
          Invalidates the specified blocks
 boolean isValidBlock(Block b)
          Is the block valid?
 boolean metaFileExists(Block b)
          Does the meta file exist for this block?
 org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface recoverAppend(Block b, long newGS, long expectedBlockLen)
          Recover a failed append to a finalized replica and returns the meta info of the replica
 void recoverClose(Block b, long newGS, long expectedBlockLen)
          Recover a failed pipeline close It bumps the replica's generation stamp and finalize it if RBW replica
 org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface recoverRbw(Block b, long newGS, long minBytesRcvd, long maxBytesRcvd)
          Recovers a RBW replica and returns the meta info of the replica
 void shutdown()
          Shutdown the FSDataset
 String toString()
          Stringifies the name of the storage
 void unfinalizeBlock(Block b)
          Unfinalizes the block previously opened for writing using writeToBlock.
 ReplicaInfo updateReplicaUnderRecovery(Block oldBlock, long recoveryId, long newLength)
          Update replica's generation stamp and length and finalize it.
 
Methods inherited from interface org.apache.hadoop.hdfs.server.datanode.metrics.FSDatasetMBean
getCapacity, getDfsUsed, getRemaining, getStorageInfo
 

Method Detail

getMetaDataLength

long getMetaDataLength(Block b)
                       throws IOException
Returns the length of the metadata file of the specified block

Parameters:
b - - the block for which the metadata length is desired
Returns:
the length of the metadata file for the specified block.
Throws:
IOException

getMetaDataInputStream

FSDatasetInterface.MetaDataInputStream getMetaDataInputStream(Block b)
                                                              throws IOException
Returns metaData of block b as an input stream (and its length)

Parameters:
b - - the block
Returns:
the metadata input stream;
Throws:
IOException

metaFileExists

boolean metaFileExists(Block b)
                       throws IOException
Does the meta file exist for this block?

Parameters:
b - - the block
Returns:
true of the metafile for specified block exits
Throws:
IOException

getLength

long getLength(Block b)
               throws IOException
Returns the specified block's on-disk length (excluding metadata)

Parameters:
b -
Returns:
the specified block's on-disk length (excluding metadta)
Throws:
IOException

getReplica

@Deprecated
Replica getReplica(long blockId)
Deprecated. 

Get reference to the replica meta info in the replicasMap. To be called from methods that are synchronized on FSDataset

Parameters:
blockId -
Returns:
replica from the replicas map

getStoredBlock

Block getStoredBlock(long blkid)
                     throws IOException
Returns:
the generation stamp stored with the block.
Throws:
IOException

getBlockInputStream

InputStream getBlockInputStream(Block b)
                                throws IOException
Returns an input stream to read the contents of the specified block

Parameters:
b -
Returns:
an input stream to read the contents of the specified block
Throws:
IOException

getBlockInputStream

InputStream getBlockInputStream(Block b,
                                long seekOffset)
                                throws IOException
Returns an input stream at specified offset of the specified block

Parameters:
b -
seekOffset -
Returns:
an input stream to read the contents of the specified block, starting at the offset
Throws:
IOException

getTmpInputStreams

FSDatasetInterface.BlockInputStreams getTmpInputStreams(Block b,
                                                        long blkoff,
                                                        long ckoff)
                                                        throws IOException
Returns an input stream at specified offset of the specified block The block is still in the tmp directory and is not finalized

Parameters:
b -
blkoff -
ckoff -
Returns:
an input stream to read the contents of the specified block, starting at the offset
Throws:
IOException

createTemporary

org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface createTemporary(Block b)
                                                                                  throws IOException
Creates a temporary replica and returns the meta information of the replica

Parameters:
b - block
Returns:
the meta info of the replica which is being written to
Throws:
IOException - if an error occurs

createRbw

org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface createRbw(Block b)
                                                                            throws IOException
Creates a RBW replica and returns the meta info of the replica

Parameters:
b - block
Returns:
the meta info of the replica which is being written to
Throws:
IOException - if an error occurs

recoverRbw

org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface recoverRbw(Block b,
                                                                             long newGS,
                                                                             long minBytesRcvd,
                                                                             long maxBytesRcvd)
                                                                             throws IOException
Recovers a RBW replica and returns the meta info of the replica

Parameters:
b - block
newGS - the new generation stamp for the replica
minBytesRcvd - the minimum number of bytes that the replica could have
maxBytesRcvd - the maximum number of bytes that the replica could have
Returns:
the meta info of the replica which is being written to
Throws:
IOException - if an error occurs

append

org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface append(Block b,
                                                                         long newGS,
                                                                         long expectedBlockLen)
                                                                         throws IOException
Append to a finalized replica and returns the meta info of the replica

Parameters:
b - block
newGS - the new generation stamp for the replica
expectedBlockLen - the number of bytes the replica is expected to have
Returns:
the meata info of the replica which is being written to
Throws:
IOException

recoverAppend

org.apache.hadoop.hdfs.server.datanode.ReplicaInPipelineInterface recoverAppend(Block b,
                                                                                long newGS,
                                                                                long expectedBlockLen)
                                                                                throws IOException
Recover a failed append to a finalized replica and returns the meta info of the replica

Parameters:
b - block
newGS - the new generation stamp for the replica
expectedBlockLen - the number of bytes the replica is expected to have
Returns:
the meta info of the replica which is being written to
Throws:
IOException

recoverClose

void recoverClose(Block b,
                  long newGS,
                  long expectedBlockLen)
                  throws IOException
Recover a failed pipeline close It bumps the replica's generation stamp and finalize it if RBW replica

Parameters:
b - block
newGS - the new generation stamp for the replica
expectedBlockLen - the number of bytes the replica is expected to have
Throws:
IOException

finalizeBlock

void finalizeBlock(Block b)
                   throws IOException
Finalizes the block previously opened for writing using writeToBlock. The block size is what is in the parameter b and it must match the amount of data written

Parameters:
b -
Throws:
IOException

unfinalizeBlock

void unfinalizeBlock(Block b)
                     throws IOException
Unfinalizes the block previously opened for writing using writeToBlock. The temporary file associated with this block is deleted.

Parameters:
b -
Throws:
IOException

getBlockReport

BlockListAsLongs getBlockReport()
Returns the block report - the full list of blocks stored

Returns:
- the block report - the full list of blocks stored

isValidBlock

boolean isValidBlock(Block b)
Is the block valid?

Parameters:
b -
Returns:
- true if the specified block is valid

invalidate

void invalidate(Block[] invalidBlks)
                throws IOException
Invalidates the specified blocks

Parameters:
invalidBlks - - the blocks to be invalidated
Throws:
IOException

checkDataDir

void checkDataDir()
                  throws org.apache.hadoop.util.DiskChecker.DiskErrorException
Check if all the data directories are healthy

Throws:
org.apache.hadoop.util.DiskChecker.DiskErrorException

toString

String toString()
Stringifies the name of the storage

Overrides:
toString in class Object

shutdown

void shutdown()
Shutdown the FSDataset


adjustCrcChannelPosition

void adjustCrcChannelPosition(Block b,
                              FSDatasetInterface.BlockWriteStreams stream,
                              int checksumSize)
                              throws IOException
Sets the file pointer of the checksum stream so that the last checksum will be overwritten

Parameters:
b - block
stream - The stream for the data file and checksum file
checksumSize - number of bytes each checksum has
Throws:
IOException

hasEnoughResource

boolean hasEnoughResource()
checks how many valid storage volumes are there in the DataNode

Returns:
true if more then minimum valid volumes left in the FSDataSet

getReplicaVisibleLength

long getReplicaVisibleLength(Block block)
                             throws IOException
Get visible length of the specified replica.

Throws:
IOException

initReplicaRecovery

ReplicaRecoveryInfo initReplicaRecovery(BlockRecoveryCommand.RecoveringBlock rBlock)
                                        throws IOException
Initialize a replica recovery.

Returns:
actual state of the replica on this data-node or null if data-node does not have the replica.
Throws:
IOException

updateReplicaUnderRecovery

ReplicaInfo updateReplicaUnderRecovery(Block oldBlock,
                                       long recoveryId,
                                       long newLength)
                                       throws IOException
Update replica's generation stamp and length and finalize it.

Throws:
IOException


Copyright © 2009 The Apache Software Foundation