org.apache.hadoop.examples.terasort
Class TeraOutputFormat

java.lang.Object
  extended by org.apache.hadoop.mapreduce.OutputFormat<K,V>
      extended by org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
          extended by org.apache.hadoop.examples.terasort.TeraOutputFormat

public class TeraOutputFormat
extends FileOutputFormat<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

An output format that writes the key and value appended together.


Nested Class Summary
static class TeraOutputFormat.TeraOutputCommitter
           
 
Field Summary
 
Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
BASE_OUTPUT_NAME, COMPRESS, COMPRESS_CODEC, COMPRESS_TYPE, OUTDIR, PART
 
Constructor Summary
TeraOutputFormat()
           
 
Method Summary
 void checkOutputSpecs(JobContext job)
          Check for validity of the output-specification for the job.
static boolean getFinalSync(JobContext job)
          Does the user want a final sync at close?
 OutputCommitter getOutputCommitter(TaskAttemptContext context)
          Get the output committer for this output format.
 RecordWriter<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> getRecordWriter(TaskAttemptContext job)
          Get the RecordWriter for the given task.
 
Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat
getCompressOutput, getDefaultWorkFile, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

TeraOutputFormat

public TeraOutputFormat()
Method Detail

getFinalSync

public static boolean getFinalSync(JobContext job)
Does the user want a final sync at close?


checkOutputSpecs

public void checkOutputSpecs(JobContext job)
                      throws InvalidJobConfException,
                             IOException
Description copied from class: OutputFormat
Check for validity of the output-specification for the job.

This is to validate the output specification for the job when it is a job is submitted. Typically checks that it does not already exist, throwing an exception when it already exists, so that output is not overwritten.

Overrides:
checkOutputSpecs in class FileOutputFormat<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
Parameters:
job - information about the job
Throws:
IOException - when output should not be attempted
InvalidJobConfException

getRecordWriter

public RecordWriter<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> getRecordWriter(TaskAttemptContext job)
                                                                                  throws IOException
Description copied from class: OutputFormat
Get the RecordWriter for the given task.

Specified by:
getRecordWriter in class FileOutputFormat<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
Parameters:
job - the information about the current task.
Returns:
a RecordWriter to write the output for the job.
Throws:
IOException

getOutputCommitter

public OutputCommitter getOutputCommitter(TaskAttemptContext context)
                                   throws IOException
Description copied from class: OutputFormat
Get the output committer for this output format. This is responsible for ensuring the output is committed correctly.

Overrides:
getOutputCommitter in class FileOutputFormat<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
Parameters:
context - the task context
Returns:
an output committer
Throws:
IOException


Copyright © 2009 The Apache Software Foundation