|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Path
to the list of inputs for the map-reduce job.
Path
with a custom InputFormat
to the list of
inputs for the map-reduce job.
Path
with a custom InputFormat
and
Mapper
to the list of inputs for the map-reduce job.
Path
to the list of inputs for the map-reduce job.
Path
with a custom InputFormat
to the list of
inputs for the map-reduce job.
Path
with a custom InputFormat
and
Mapper
to the list of inputs for the map-reduce job.
Mapper
class to the chain mapper.
Mapper
class to the chain reducer.
InputFormat
that tries to deduce the types of the input files
automatically.BaileyBorweinPlouffe.BbpMapper
.BaileyBorweinPlouffe.BbpInputFormat
.BinaryComparable
keys using a configurable part of
the bytes array returned by BinaryComparable.getBytes()
.BinaryComparable
keys using a configurable part of
the bytes array returned by BinaryComparable.getBytes()
.OutputCommitter.commitJob(JobContext)
or
OutputCommitter.abortJob(JobContext, int)
instead.
OutputCommitter.commitJob(org.apache.hadoop.mapreduce.JobContext)
or OutputCommitter.abortJob(org.apache.hadoop.mapreduce.JobContext, org.apache.hadoop.mapreduce.JobStatus.State)
instead.
OutputCommitter.commitJob(JobContext)
or
OutputCommitter.abortJob(JobContext, JobStatus.State)
instead.
JobClient
.
RecordWriter
to future operations.
InputSplit
to future operations.
RecordWriter
to future operations.
Cluster
.
RecordWriter
to future operations.
RecordWriter
to future operations.
MultiFilterRecordReader.emit(org.apache.hadoop.mapred.join.TupleWritable)
every Tuple from the
collector (the outer join of child RRs).
MultiFilterRecordReader.emit(org.apache.hadoop.mapreduce.lib.join.TupleWritable)
every Tuple from the
collector (the outer join of child RRs).
InputFormat
that returns CombineFileSplit
's
in InputFormat.getSplits(JobConf, int)
method.InputFormat
that returns CombineFileSplit
's in
InputFormat.getSplits(JobContext)
method.CombineFileSplit
.CombineFileSplit
.JobConf
.
JobConf
.
JobConf
.
Configuration
.
Counter
s that logically belong together.Group
of counters, comprising of counters from a particular
counter Enum
class.CombineFileInputFormat.createPool(List)
.
CombineFileInputFormat.createPool(PathFilter...)
.
DBWritable
.Job
.InputFormat
.InputFormat
s.OutputCommitter
that commits files specified
in job output directory i.e.OutputCommitter
that commits files specified
in job output directory i.e.OutputFormat
.OutputFormat
s that read from FileSystem
s.FilterRecordWriter
is a convenience wrapper
class that implements RecordWriter
.FilterRecordWriter
is a convenience wrapper
class that extends the RecordWriter
.UTF8ByteArrayUtils.findByte(byte[], int,
int, byte)
UTF8ByteArrayUtils.findBytes(byte[], int,
int, byte[])
UTF8ByteArrayUtils.findNthByte(byte[], int,
int, byte, int)
UTF8ByteArrayUtils.findNthByte(byte[],
byte, int)
StreamKeyValUtil.findTab(byte[], int, int)
StreamKeyValUtil.findTab(byte[])
DataInput
.
DataOutput
.
TypedBytesInput
.
DataInput
.
TypedBytesOutput
.
DataOutput
.
TypedBytesInput
.
DataInput
.
TypedBytesOutput
.
DataOutput
.
SkipBadRecords.COUNTER_MAP_PROCESSED_RECORDS
is incremented
by MapRunner after invoking the map function.
SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS
is incremented
by framework after invoking the reduce function.
Counters.Group.getCounter(String)
instead
Counters.Counter
of the given group with the given name.
Counters.Counter
of the given group with the given name.
Counter
for the given counterName
.
Counter
for the given groupName
and
counterName
.
RawComparator
comparator for
grouping keys of inputs to the reduce.
InputFormat
implementation for the map-reduce job,
defaults to TextInputFormat
if not specified explicity.
InputFormat
class for the job.
Path
s for the map-reduce job.
Path
s for the map-reduce job.
InputSplit
object for a map.
InputWriter
class.
RunningJob
object to track an ongoing job.
JobClient.getJob(JobID)
.
RunningJob.getID()
.
JobID
object that this task attempt belongs to
JobID
object that this tip belongs to
JobPriority
for this job.
JobTracker
,
as JobTracker.State
SequenceFileRecordReader.next(Object, Object)
..
KeyFieldBasedComparator
options
KeyFieldBasedComparator
options
KeyFieldBasedPartitioner
options
KeyFieldBasedPartitioner
options
InputSplit
.
WrappedMapper.Context
for custom implementations.
CompressionCodec
for compressing the map outputs.
Mapper
class for the job.
Mapper
class for the job.
MapRunnable
class for the job.
true
.
JobClient.getMapTaskReports(JobID)
mapreduce.map.maxattempts
property.
mapred.map.max.attempts
property.
JobTracker
mapreduce.reduce.maxattempts
property.
mapred.reduce.max.attempts
property.
JobConf.getMemoryForMapTask()
and
JobConf.getMemoryForReduceTask()
OutputCommitter
implementation for the map-reduce job,
defaults to FileOutputCommitter
if not specified explicitly.
OutputCommitter
for the task-attempt.
SequenceFile.CompressionType
for the output SequenceFile
.
SequenceFile.CompressionType
for the output SequenceFile
.
CompressionCodec
for compressing the job outputs.
CompressionCodec
for compressing the job outputs.
OutputFormat
implementation for the map-reduce job,
defaults to TextOutputFormat
if not specified explicity.
OutputFormat
class for the job.
RawComparator
comparator used to compare keys.
Path
to the output directory for the map-reduce job.
Path
to the output directory for the map-reduce job.
OutputReader
class.
WritableComparable
comparator for
grouping keys of inputs to the reduce.
Object.hashCode()
to partition.
BinaryComparable.getBytes()
to partition.
Object.hashCode()
to partition.
Partitioner
used to partition Mapper
-outputs
to be sent to the Reducer
s.
Partitioner
class for the job.
Path
for a file that is unique for
the task within the job output directory.
Path
for a file that is unique for
the task within the job output directory.
RecordReader
consumed i.e.
RecordReader
for the given InputSplit
.
RecordReader
for the given InputSplit
.
RecordWriter
for the given job.
RecordWriter
for the given job.
RecordWriter
for the given task.
RecordWriter
for the given task.
Reducer
class for the job.
Reducer
class for the job.
WrappedReducer.Context
for custom implementations.
true
.
JobClient.getReduceTaskReports(JobID)
TaskType
SequenceFile
SequenceFile
SequenceFile
SequenceFile
RawComparator
comparator used to compare keys.
true
.
FileInputFormat.listStatus(JobConf)
when
they're too big.
TaskCompletionEvent.getTaskAttemptId()
instead.
TaskID
object that this task attempt belongs to
TaskID.getTaskIDsPattern(String, Integer, TaskType,
Integer)
TaskType
corresponding to the character
JobTracker
SequenceFileRecordReader.next(Object, Object)
..
Path
to the task's temporary output directory
for the map-reduce job
Path
to the task's temporary output directory
for the map-reduce job
StreamJob.run(String[])
instead.
Object.hashCode()
.Object.hashCode()
.Enum
type, by the specified amount.
JobTracker
.
InputFormat
describes the input-specification for a
Map-Reduce job.InputFormat
describes the input-specification for a
Map-Reduce job.TotalOrderPartitioner
.InputFormat
.InputSplit
represents the data to be processed by an
individual Mapper
.InputSplit
represents the data to be processed by an
individual Mapper
.Mapper
that swaps keys and values.Mapper
that swaps keys and values.JobClient
is the primary interface for the user-job to interact
with the JobTracker
.JobConf
, and connect to the
default JobTracker
.
Configuration
,
and connect to the default JobTracker
.
KeyFieldBasedComparator
.KeyFieldBasedComparator
.InputFormat
for plain text files.InputFormat
for plain text files.RunningJob.killTask(TaskAttemptID, boolean)
Reducer
that sums long values.map(...)
methods of the Mappers in the chain.
Mapper
.OutputFormat
that writes MapFile
s.OutputFormat
that writes
MapFile
s.Context
passed on to the Mapper
implementations.Level
for the map task.
Level
for the reduce task.
JobConf.MAPRED_MAP_TASK_ENV
or
JobConf.MAPRED_REDUCE_TASK_ENV
JobConf.MAPRED_MAP_TASK_JAVA_OPTS
or
JobConf.MAPRED_REDUCE_TASK_JAVA_OPTS
JobConf.MAPRED_JOB_MAP_MEMORY_MB_PROPERTY
and
JobConf.MAPRED_JOB_REDUCE_MEMORY_MB_PROPERTY
JobConf.MAPRED_MAP_TASK_ULIMIT
or
JobConf.MAPRED_REDUCE_TASK_ULIMIT
Mapper
and Reducer
implementations.Mapper
s.MapRunnable
implementation.MarkableIterator
is a wrapper iterator class that
implements the MarkableIteratorInterface
.InputFormat
that returns MultiFileSplit
's
in MultiFileInputFormat.getSplits(JobConf, int)
method.MultiFileWordCount.MapClass
.CombineFileInputFormat
, one should extend it, to return a
(custom) RecordReader
.InputFormat
and Mapper
for each pathInputFormat
and Mapper
for each pathOutputCollector
passed to
the map()
and reduce()
methods of the
Mapper
and Reducer
implementations.DBRecordReader.nextKeyValue()
Job
instead.<key, value>
pairs output by Mapper
s
and Reducer
s.OutputCommitter
describes the commit of task output for a
Map-Reduce job.OutputCommitter
describes the commit of task output for a
Map-Reduce job.OutputFormat
describes the output-specification for a
Map-Reduce job.OutputFormat
describes the output-specification for a
Map-Reduce job.Utils.OutputFileUtils.OutputLogFilter
instead.Type.BOOL
code.
Type.BYTE
code.
Type.BYTES
code.
Type.DOUBLE
code.
ResultSet
.
Type.FLOAT
code.
Type.INT
code.
StreamKeyValUtil.readLine(LineReader, Text)
Type.LIST
code.
Type.LONG
code.
Type.MAP
code.
Type.MAP
code.
Type.BOOL
code.
Type.BYTE
code.
Type.BYTES
code.
Type.DOUBLE
code.
Type.FLOAT
code.
Type.INT
code.
Type.LIST
code.
Type.LONG
code.
Type.MAP
code.
Type.STRING
code.
Type.VECTOR
code.
Type.STRING
code.
Type
.
Type.VECTOR
code.
Type.VECTOR
code.
RecordReader
reads <key, value> pairs from an
InputSplit
.Mapper
.RecordWriter
writes the output <key, value> pairs
to an output file.RecordWriter
writes the output <key, value> pairs
to an output file.reduce(...)
method of the Reducer with the
map(...)
methods of the Mappers in the chain.
Reducer
.Iterator
to iterate over values for a given group of records.Context
passed on to the Reducer
implementations.Mapper
that extracts text matching a regular expression.Mapper
that extracts text matching a regular expression.Reducer.run(org.apache.hadoop.mapreduce.Reducer.Context)
method to
control how the reduce task works.
DumpTypedBytes
.
LoadTypedBytes
.
RunningJob
is the user-interface to query for details on a
running Map-Reduce job.OutputFormat
that writes keys, values to
SequenceFile
s in binary(raw) formatOutputFormat
that writes keys,
values to SequenceFile
s in binary(raw) formatInputFormat
for SequenceFile
s.InputFormat
for SequenceFile
s.OutputFormat
that writes SequenceFile
s.OutputFormat
that writes SequenceFile
s.RecordReader
for SequenceFile
s.RecordReader
for SequenceFile
s.SkipBadRecords.COUNTER_MAP_PROCESSED_RECORDS
is incremented
by MapRunner after invoking the map function.
SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS
is incremented
by framework after invoking the reduce function.
Reducer.reduce(Object, Iterable,
org.apache.hadoop.mapreduce.Reducer.Context)
InputFormat
implementation for the map-reduce job.
InputFormat
for the job.
Path
s as the list of inputs
for the map-reduce job.
Path
s as the list of inputs
for the map-reduce job.
InputWriter
class.
JobPriority
for this job.
KeyFieldBasedComparator
options used to compare keys.
KeyFieldBasedComparator
options used to compare keys.
KeyFieldBasedPartitioner
options used for
Partitioner
KeyFieldBasedPartitioner
options used for
Partitioner
bytes[offset:]
in Python syntax.
CompressionCodec
for the map outputs.
Mapper
class for the job.
Mapper
for the job.
MapRunnable
class for the job.
JobConf.setMemoryForMapTask(long mem)
and
Use JobConf.setMemoryForReduceTask(long mem)
bytes[left:(right+1)]
in Python syntax.
OutputCommitter
implementation for the map-reduce job.
SequenceFile.CompressionType
for the output SequenceFile
.
SequenceFile.CompressionType
for the output SequenceFile
.
CompressionCodec
to be used to compress job outputs.
CompressionCodec
to be used to compress job outputs.
OutputFormat
implementation for the map-reduce job.
OutputFormat
for the job.
RawComparator
comparator used to compare keys.
Path
of the output directory for the map-reduce job.
Path
of the output directory for the map-reduce job.
OutputReader
class.
RawComparator
comparator for
grouping keys in the input to the reduce.
Partitioner
class used to partition
Mapper
-outputs to be sent to the Reducer
s.
Partitioner
for the job.
Reducer
class to the chain job.
Reducer
class for the job.
Reducer
for the job.
bytes[:(offset+1)]
in Python syntax.
SequenceFile
SequenceFile
SequenceFile
SequenceFile
Reducer
.
TaskCompletionEvent.setTaskAttemptId(TaskAttemptID)
instead.
StreamKeyValUtil.splitKeyVal(byte[], int, int, Text, Text,
int, int)
StreamKeyValUtil.splitKeyVal(byte[], int, int, Text, Text, int)
StreamKeyValUtil.splitKeyVal(byte[], Text, Text, int, int)
StreamKeyValUtil.splitKeyVal(byte[], Text, Text, int)
StreamJob.setConf(Configuration)
and
run with StreamJob.run(String[])
.
Submitter.runJob(JobConf)
TaskID
.
TaskAttemptID.TaskAttemptID(String, int, TaskType, int, int)
.
TaskID
.
TaskID.TaskID(String, int, TaskType, int)
TaskID.TaskID(org.apache.hadoop.mapreduce.JobID, TaskType,
int)
JobID
.
JobID
.
InputFormat
for plain text files.InputFormat
for plain text files.OutputFormat
that writes plain text files.OutputFormat
that writes plain text files.Mapper
that maps text values into Writable
s.Writable
s.UTF8ByteArrayUtils
and
StreamKeyValUtil
insteadMapper
which wraps a given one to allow custom
WrappedMapper.Context
implementations.Reducer
which wraps a given one to allow for custom
WrappedReducer.Context
implementations.PreparedStatement
.
out
.
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |