|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use JobConf | |
---|---|
org.apache.hadoop.contrib.index.example | |
org.apache.hadoop.contrib.index.mapred | |
org.apache.hadoop.contrib.utils.join | |
org.apache.hadoop.mapred | A software framework for easily writing applications which process vast amounts of data (multi-terabyte data-sets) parallelly on large clusters (thousands of nodes) built of commodity hardware in a reliable, fault-tolerant manner. |
org.apache.hadoop.mapred.jobcontrol | Utilities for managing dependent jobs. |
org.apache.hadoop.mapred.join | Given a set of sorted datasets keyed with the same class and yielding equal partitions, it is possible to effect a join of those datasets prior to the map. |
org.apache.hadoop.mapred.lib | Library of generally useful mappers, reducers, and partitioners. |
org.apache.hadoop.mapred.lib.aggregate | Classes for performing various counting and aggregations. |
org.apache.hadoop.mapred.lib.db | org.apache.hadoop.mapred.lib.db Package |
org.apache.hadoop.mapred.pipes | Hadoop Pipes allows C++ code to use Hadoop DFS and map/reduce. |
org.apache.hadoop.mapreduce.security | |
org.apache.hadoop.streaming | Hadoop Streaming is a utility which allows users to create and run Map-Reduce jobs with any executables (e.g. |
Uses of JobConf in org.apache.hadoop.contrib.index.example |
---|
Methods in org.apache.hadoop.contrib.index.example with parameters of type JobConf | |
---|---|
void |
LineDocLocalAnalysis.configure(JobConf job)
|
void |
IdentityLocalAnalysis.configure(JobConf job)
|
RecordReader<DocumentID,LineDocTextAndOp> |
LineDocInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
Uses of JobConf in org.apache.hadoop.contrib.index.mapred |
---|
Methods in org.apache.hadoop.contrib.index.mapred with parameters of type JobConf | |
---|---|
void |
IndexUpdateCombiner.configure(JobConf job)
|
void |
IndexUpdateMapper.configure(JobConf job)
|
void |
IndexUpdateReducer.configure(JobConf job)
|
void |
IndexUpdatePartitioner.configure(JobConf job)
|
RecordWriter<Shard,org.apache.hadoop.io.Text> |
IndexUpdateOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem fs,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
Uses of JobConf in org.apache.hadoop.contrib.utils.join |
---|
Fields in org.apache.hadoop.contrib.utils.join declared as JobConf | |
---|---|
protected JobConf |
DataJoinMapperBase.job
|
protected JobConf |
DataJoinReducerBase.job
|
Methods in org.apache.hadoop.contrib.utils.join that return JobConf | |
---|---|
static JobConf |
DataJoinJob.createDataJoinJob(String[] args)
|
Methods in org.apache.hadoop.contrib.utils.join with parameters of type JobConf | |
---|---|
TaggedMapOutput |
TaggedMapOutput.clone(JobConf job)
|
void |
JobBase.configure(JobConf job)
Initializes a new instance from a JobConf . |
void |
DataJoinMapperBase.configure(JobConf job)
|
void |
DataJoinReducerBase.configure(JobConf job)
|
static boolean |
DataJoinJob.runJob(JobConf job)
Submit/run a map/reduce job. |
Uses of JobConf in org.apache.hadoop.mapred |
---|
Methods in org.apache.hadoop.mapred that return JobConf | |
---|---|
JobConf |
JobContext.getJobConf()
Get the job Configuration |
JobConf |
TaskAttemptContext.getJobConf()
|
Methods in org.apache.hadoop.mapred with parameters of type JobConf | |
---|---|
static void |
FileInputFormat.addInputPath(JobConf conf,
org.apache.hadoop.fs.Path path)
Add a Path to the list of inputs for the map-reduce job. |
static void |
FileInputFormat.addInputPaths(JobConf conf,
String commaSeparatedPaths)
Add the given comma separated paths to the list of inputs for the map-reduce job. |
void |
FileOutputFormat.checkOutputSpecs(org.apache.hadoop.fs.FileSystem ignored,
JobConf job)
|
void |
SequenceFileAsBinaryOutputFormat.checkOutputSpecs(org.apache.hadoop.fs.FileSystem ignored,
JobConf job)
|
void |
OutputFormat.checkOutputSpecs(org.apache.hadoop.fs.FileSystem ignored,
JobConf job)
Check for validity of the output-specification for the job. |
void |
JobConfigurable.configure(JobConf job)
Initializes a new instance from a JobConf . |
void |
MapReduceBase.configure(JobConf job)
Default implementation that does nothing. |
void |
TextInputFormat.configure(JobConf conf)
|
void |
KeyValueTextInputFormat.configure(JobConf conf)
|
void |
MapRunner.configure(JobConf job)
|
static boolean |
FileOutputFormat.getCompressOutput(JobConf conf)
Is the job output compressed? |
static org.apache.hadoop.fs.PathFilter |
FileInputFormat.getInputPathFilter(JobConf conf)
Get a PathFilter instance of the filter set for the input paths. |
static org.apache.hadoop.fs.Path[] |
FileInputFormat.getInputPaths(JobConf conf)
Get the list of input Path s for the map-reduce job. |
static org.apache.hadoop.io.SequenceFile.CompressionType |
SequenceFileOutputFormat.getOutputCompressionType(JobConf conf)
Get the SequenceFile.CompressionType for the output SequenceFile . |
static Class<? extends org.apache.hadoop.io.compress.CompressionCodec> |
FileOutputFormat.getOutputCompressorClass(JobConf conf,
Class<? extends org.apache.hadoop.io.compress.CompressionCodec> defaultValue)
Get the CompressionCodec for compressing the job outputs. |
static org.apache.hadoop.fs.Path |
FileOutputFormat.getOutputPath(JobConf conf)
Get the Path to the output directory for the map-reduce job. |
static org.apache.hadoop.fs.Path |
FileOutputFormat.getPathForCustomFile(JobConf conf,
String name)
Helper function to generate a Path for a file that is unique for
the task within the job output directory. |
RecordReader<K,V> |
SequenceFileInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> |
SequenceFileAsTextInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
RecordReader<K,V> |
InputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
Get the RecordReader for the given InputSplit . |
RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> |
TextInputFormat.getRecordReader(InputSplit genericSplit,
JobConf job,
Reporter reporter)
|
RecordReader<K,V> |
SequenceFileInputFilter.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
Create a record reader for the given split |
RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> |
KeyValueTextInputFormat.getRecordReader(InputSplit genericSplit,
JobConf job,
Reporter reporter)
|
abstract RecordReader<K,V> |
MultiFileInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
abstract RecordReader<K,V> |
FileInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
RecordReader<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable> |
SequenceFileAsBinaryInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
RecordWriter<K,V> |
TextOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
abstract RecordWriter<K,V> |
FileOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
RecordWriter<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable> |
SequenceFileAsBinaryOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
RecordWriter<K,V> |
OutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
Get the RecordWriter for the given job. |
RecordWriter<K,V> |
SequenceFileOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
RecordWriter<org.apache.hadoop.io.WritableComparable,org.apache.hadoop.io.Writable> |
MapFileOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
static Class<? extends org.apache.hadoop.io.WritableComparable> |
SequenceFileAsBinaryOutputFormat.getSequenceFileOutputKeyClass(JobConf conf)
Get the key class for the SequenceFile |
static Class<? extends org.apache.hadoop.io.Writable> |
SequenceFileAsBinaryOutputFormat.getSequenceFileOutputValueClass(JobConf conf)
Get the value class for the SequenceFile |
InputSplit[] |
InputFormat.getSplits(JobConf job,
int numSplits)
Logically split the set of input files for the job. |
InputSplit[] |
MultiFileInputFormat.getSplits(JobConf job,
int numSplits)
|
InputSplit[] |
FileInputFormat.getSplits(JobConf job,
int numSplits)
Splits files returned by FileInputFormat.listStatus(JobConf) when
they're too big. |
static JobClient.TaskStatusFilter |
JobClient.getTaskOutputFilter(JobConf job)
Get the task output filter out of the JobConf. |
static org.apache.hadoop.fs.Path |
FileOutputFormat.getTaskOutputPath(JobConf conf,
String name)
Helper function to create the task's temporary output directory and return the path to the task's output file. |
static String |
FileOutputFormat.getUniqueName(JobConf conf,
String name)
Helper function to generate a name that is unique for the task. |
static org.apache.hadoop.fs.Path |
FileOutputFormat.getWorkOutputPath(JobConf conf)
Get the Path to the task's temporary output directory
for the map-reduce job
Tasks' Side-Effect Files |
void |
JobClient.init(JobConf conf)
Connect to the default JobTracker . |
protected org.apache.hadoop.fs.FileStatus[] |
SequenceFileInputFormat.listStatus(JobConf job)
|
protected org.apache.hadoop.fs.FileStatus[] |
FileInputFormat.listStatus(JobConf job)
List input directories. |
boolean |
JobClient.monitorAndPrintJob(JobConf conf,
RunningJob job)
Monitor a job and print status in real-time as progress is made and tasks fail. |
static RunningJob |
JobClient.runJob(JobConf job)
Utility that submits a job, then polls for progress until the job is complete. |
static void |
FileOutputFormat.setCompressOutput(JobConf conf,
boolean compress)
Set whether the output of the job is compressed. |
static void |
FileInputFormat.setInputPathFilter(JobConf conf,
Class<? extends org.apache.hadoop.fs.PathFilter> filter)
Set a PathFilter to be applied to the input paths for the map-reduce job. |
static void |
FileInputFormat.setInputPaths(JobConf conf,
org.apache.hadoop.fs.Path... inputPaths)
Set the array of Path s as the list of inputs
for the map-reduce job. |
static void |
FileInputFormat.setInputPaths(JobConf conf,
String commaSeparatedPaths)
Sets the given comma separated paths as the list of inputs for the map-reduce job. |
static void |
SequenceFileOutputFormat.setOutputCompressionType(JobConf conf,
org.apache.hadoop.io.SequenceFile.CompressionType style)
Set the SequenceFile.CompressionType for the output SequenceFile . |
static void |
FileOutputFormat.setOutputCompressorClass(JobConf conf,
Class<? extends org.apache.hadoop.io.compress.CompressionCodec> codecClass)
Set the CompressionCodec to be used to compress job outputs. |
static void |
FileOutputFormat.setOutputPath(JobConf conf,
org.apache.hadoop.fs.Path outputDir)
Set the Path of the output directory for the map-reduce job. |
static void |
SequenceFileAsBinaryOutputFormat.setSequenceFileOutputKeyClass(JobConf conf,
Class<?> theClass)
Set the key class for the SequenceFile |
static void |
SequenceFileAsBinaryOutputFormat.setSequenceFileOutputValueClass(JobConf conf,
Class<?> theClass)
Set the value class for the SequenceFile |
static void |
SkipBadRecords.setSkipOutputPath(JobConf conf,
org.apache.hadoop.fs.Path path)
Set the directory to which skipped records are written. |
static void |
JobClient.setTaskOutputFilter(JobConf job,
JobClient.TaskStatusFilter newValue)
Modify the JobConf to set the task output filter. |
RunningJob |
JobClient.submitJob(JobConf conf)
Submit a job to the MR system. |
Constructors in org.apache.hadoop.mapred with parameters of type JobConf | |
---|---|
FileSplit(org.apache.hadoop.fs.Path file,
long start,
long length,
JobConf conf)
Deprecated. |
|
JobClient(JobConf conf)
Build a job client with the given JobConf , and connect to the
default JobTracker . |
|
MultiFileSplit(JobConf job,
org.apache.hadoop.fs.Path[] files,
long[] lengths)
|
Uses of JobConf in org.apache.hadoop.mapred.jobcontrol |
---|
Methods in org.apache.hadoop.mapred.jobcontrol that return JobConf | |
---|---|
JobConf |
Job.getJobConf()
|
Methods in org.apache.hadoop.mapred.jobcontrol with parameters of type JobConf | |
---|---|
void |
Job.setJobConf(JobConf jobConf)
Set the mapred job conf for this job. |
Constructors in org.apache.hadoop.mapred.jobcontrol with parameters of type JobConf | |
---|---|
Job(JobConf conf)
|
|
Job(JobConf jobConf,
ArrayList<?> dependingJobs)
Construct a job. |
Uses of JobConf in org.apache.hadoop.mapred.join |
---|
Methods in org.apache.hadoop.mapred.join with parameters of type JobConf | |
---|---|
ComposableRecordReader<K,TupleWritable> |
CompositeInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
Construct a CompositeRecordReader for the children of this InputFormat as defined in the init expression. |
ComposableRecordReader<K,V> |
ComposableInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
InputSplit[] |
CompositeInputFormat.getSplits(JobConf job,
int numSplits)
Build a CompositeInputSplit from the child InputFormats by assigning the ith split from each child to the ith composite split. |
void |
CompositeInputFormat.setFormat(JobConf job)
Interpret a given string as a composite expression. |
Constructors in org.apache.hadoop.mapred.join with parameters of type JobConf | |
---|---|
JoinRecordReader(int id,
JobConf conf,
int capacity,
Class<? extends org.apache.hadoop.io.WritableComparator> cmpcl)
|
|
MultiFilterRecordReader(int id,
JobConf conf,
int capacity,
Class<? extends org.apache.hadoop.io.WritableComparator> cmpcl)
|
Uses of JobConf in org.apache.hadoop.mapred.lib |
---|
Fields in org.apache.hadoop.mapred.lib declared as JobConf | |
---|---|
protected JobConf |
CombineFileRecordReader.jc
|
Methods in org.apache.hadoop.mapred.lib that return JobConf | |
---|---|
JobConf |
CombineFileSplit.getJob()
|
Methods in org.apache.hadoop.mapred.lib with parameters of type JobConf | ||
---|---|---|
static void |
MultipleInputs.addInputPath(JobConf conf,
org.apache.hadoop.fs.Path path,
Class<? extends InputFormat> inputFormatClass)
Add a Path with a custom InputFormat to the list of
inputs for the map-reduce job. |
|
static void |
MultipleInputs.addInputPath(JobConf conf,
org.apache.hadoop.fs.Path path,
Class<? extends InputFormat> inputFormatClass,
Class<? extends Mapper> mapperClass)
Add a Path with a custom InputFormat and
Mapper to the list of inputs for the map-reduce job. |
|
static
|
ChainReducer.addMapper(JobConf job,
Class<? extends Mapper<K1,V1,K2,V2>> klass,
Class<? extends K1> inputKeyClass,
Class<? extends V1> inputValueClass,
Class<? extends K2> outputKeyClass,
Class<? extends V2> outputValueClass,
boolean byValue,
JobConf mapperConf)
Adds a Mapper class to the chain job's JobConf. |
|
static
|
ChainMapper.addMapper(JobConf job,
Class<? extends Mapper<K1,V1,K2,V2>> klass,
Class<? extends K1> inputKeyClass,
Class<? extends V1> inputValueClass,
Class<? extends K2> outputKeyClass,
Class<? extends V2> outputValueClass,
boolean byValue,
JobConf mapperConf)
Adds a Mapper class to the chain job's JobConf. |
|
static void |
MultipleOutputs.addMultiNamedOutput(JobConf conf,
String namedOutput,
Class<? extends OutputFormat> outputFormatClass,
Class<?> keyClass,
Class<?> valueClass)
Adds a multi named output for the job. |
|
static void |
MultipleOutputs.addNamedOutput(JobConf conf,
String namedOutput,
Class<? extends OutputFormat> outputFormatClass,
Class<?> keyClass,
Class<?> valueClass)
Adds a named output for the job. |
|
void |
FilterOutputFormat.checkOutputSpecs(org.apache.hadoop.fs.FileSystem ignored,
JobConf job)
|
|
void |
NullOutputFormat.checkOutputSpecs(org.apache.hadoop.fs.FileSystem ignored,
JobConf job)
|
|
void |
LazyOutputFormat.checkOutputSpecs(org.apache.hadoop.fs.FileSystem ignored,
JobConf job)
|
|
void |
NLineInputFormat.configure(JobConf conf)
|
|
void |
ChainReducer.configure(JobConf job)
Configures the ChainReducer, the Reducer and all the Mappers in the chain. |
|
void |
MultithreadedMapRunner.configure(JobConf jobConf)
|
|
void |
FieldSelectionMapReduce.configure(JobConf job)
|
|
void |
KeyFieldBasedPartitioner.configure(JobConf job)
|
|
void |
TotalOrderPartitioner.configure(JobConf job)
|
|
void |
RegexMapper.configure(JobConf job)
|
|
void |
BinaryPartitioner.configure(JobConf job)
|
|
void |
KeyFieldBasedComparator.configure(JobConf job)
|
|
void |
HashPartitioner.configure(JobConf job)
|
|
void |
ChainMapper.configure(JobConf job)
Configures the ChainMapper and all the Mappers in the chain. |
|
protected void |
CombineFileInputFormat.createPool(JobConf conf,
List<org.apache.hadoop.fs.PathFilter> filters)
Deprecated. Use CombineFileInputFormat.createPool(List) . |
|
protected void |
CombineFileInputFormat.createPool(JobConf conf,
org.apache.hadoop.fs.PathFilter... filters)
Deprecated. Use CombineFileInputFormat.createPool(PathFilter...) . |
|
protected abstract RecordWriter<K,V> |
MultipleOutputFormat.getBaseRecordWriter(org.apache.hadoop.fs.FileSystem fs,
JobConf job,
String name,
org.apache.hadoop.util.Progressable arg3)
|
|
protected RecordWriter<K,V> |
MultipleTextOutputFormat.getBaseRecordWriter(org.apache.hadoop.fs.FileSystem fs,
JobConf job,
String name,
org.apache.hadoop.util.Progressable arg3)
|
|
protected RecordWriter<K,V> |
MultipleSequenceFileOutputFormat.getBaseRecordWriter(org.apache.hadoop.fs.FileSystem fs,
JobConf job,
String name,
org.apache.hadoop.util.Progressable arg3)
|
|
static boolean |
MultipleOutputs.getCountersEnabled(JobConf conf)
Returns if the counters for the named outputs are enabled or not. |
|
protected String |
MultipleOutputFormat.getInputFileBasedOutputFileName(JobConf job,
String name)
Generate the outfile name based on a given anme and the input file name. |
|
static Class<? extends OutputFormat> |
MultipleOutputs.getNamedOutputFormatClass(JobConf conf,
String namedOutput)
Returns the named output OutputFormat. |
|
static Class<? extends org.apache.hadoop.io.WritableComparable> |
MultipleOutputs.getNamedOutputKeyClass(JobConf conf,
String namedOutput)
Returns the key class for a named output. |
|
static List<String> |
MultipleOutputs.getNamedOutputsList(JobConf conf)
Returns list of channel names. |
|
static Class<? extends org.apache.hadoop.io.Writable> |
MultipleOutputs.getNamedOutputValueClass(JobConf conf,
String namedOutput)
Returns the value class for a named output. |
|
RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> |
NLineInputFormat.getRecordReader(InputSplit genericSplit,
JobConf job,
Reporter reporter)
|
|
abstract RecordReader<K,V> |
CombineFileInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
This is not implemented yet. |
|
RecordWriter<K,V> |
MultipleOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem fs,
JobConf job,
String name,
org.apache.hadoop.util.Progressable arg3)
Create a composite record writer that can write key/value data to different output files |
|
RecordWriter<K,V> |
FilterOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
|
RecordWriter<K,V> |
NullOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
|
RecordWriter<K,V> |
LazyOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem ignored,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
|
|
InputSplit[] |
NLineInputFormat.getSplits(JobConf job,
int numSplits)
Logically splits the set of input files for the job, splits N lines of the input as one split. |
|
InputSplit[] |
CombineFileInputFormat.getSplits(JobConf job,
int numSplits)
|
|
static boolean |
MultipleOutputs.isMultiNamedOutput(JobConf conf,
String namedOutput)
Returns if a named output is multiple. |
|
static void |
MultipleOutputs.setCountersEnabled(JobConf conf,
boolean enabled)
Enables or disables counters for the named outputs. |
|
static void |
LazyOutputFormat.setOutputFormatClass(JobConf job,
Class<? extends OutputFormat> theClass)
Set the underlying output format for LazyOutputFormat. |
|
static
|
ChainReducer.setReducer(JobConf job,
Class<? extends Reducer<K1,V1,K2,V2>> klass,
Class<? extends K1> inputKeyClass,
Class<? extends V1> inputValueClass,
Class<? extends K2> outputKeyClass,
Class<? extends V2> outputValueClass,
boolean byValue,
JobConf reducerConf)
Sets the Reducer class to the chain job's JobConf. |
|
static
|
InputSampler.writePartitionFile(JobConf job,
InputSampler.Sampler<K,V> sampler)
|
Constructors in org.apache.hadoop.mapred.lib with parameters of type JobConf | |
---|---|
CombineFileRecordReader(JobConf job,
CombineFileSplit split,
Reporter reporter,
Class<RecordReader<K,V>> rrClass)
A generic RecordReader that can hand out different recordReaders for each chunk in the CombineFileSplit. |
|
CombineFileSplit(JobConf job,
org.apache.hadoop.fs.Path[] files,
long[] lengths)
|
|
CombineFileSplit(JobConf job,
org.apache.hadoop.fs.Path[] files,
long[] start,
long[] lengths,
String[] locations)
|
|
InputSampler(JobConf conf)
|
|
MultipleOutputs(JobConf job)
Creates and initializes multiple named outputs support, it should be instantiated in the Mapper/Reducer configure method. |
Uses of JobConf in org.apache.hadoop.mapred.lib.aggregate |
---|
Methods in org.apache.hadoop.mapred.lib.aggregate that return JobConf | |
---|---|
static JobConf |
ValueAggregatorJob.createValueAggregatorJob(String[] args)
Create an Aggregate based map/reduce job. |
static JobConf |
ValueAggregatorJob.createValueAggregatorJob(String[] args,
Class<? extends ValueAggregatorDescriptor>[] descriptors)
|
Methods in org.apache.hadoop.mapred.lib.aggregate with parameters of type JobConf | |
---|---|
void |
ValueAggregatorCombiner.configure(JobConf job)
Combiner does not need to configure. |
void |
ValueAggregatorJobBase.configure(JobConf job)
|
void |
ValueAggregatorBaseDescriptor.configure(JobConf job)
get the input file name. |
void |
UserDefinedValueAggregatorDescriptor.configure(JobConf job)
Do nothing. |
void |
ValueAggregatorDescriptor.configure(JobConf job)
Configure the object |
static void |
ValueAggregatorJob.setAggregatorDescriptors(JobConf job,
Class<? extends ValueAggregatorDescriptor>[] descriptors)
|
Constructors in org.apache.hadoop.mapred.lib.aggregate with parameters of type JobConf | |
---|---|
UserDefinedValueAggregatorDescriptor(String className,
JobConf job)
|
Uses of JobConf in org.apache.hadoop.mapred.lib.db |
---|
Methods in org.apache.hadoop.mapred.lib.db with parameters of type JobConf | |
---|---|
void |
DBOutputFormat.checkOutputSpecs(org.apache.hadoop.fs.FileSystem filesystem,
JobConf job)
Check for validity of the output-specification for the job. |
void |
DBInputFormat.configure(JobConf job)
Initializes a new instance from a JobConf . |
static void |
DBConfiguration.configureDB(JobConf job,
String driverClass,
String dbUrl)
Sets the DB access related fields in the JobConf. |
static void |
DBConfiguration.configureDB(JobConf job,
String driverClass,
String dbUrl,
String userName,
String passwd)
Sets the DB access related fields in the JobConf. |
RecordReader<org.apache.hadoop.io.LongWritable,T> |
DBInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
Get the RecordReader for the given InputSplit . |
RecordWriter<K,V> |
DBOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem filesystem,
JobConf job,
String name,
org.apache.hadoop.util.Progressable progress)
Get the RecordWriter for the given job. |
InputSplit[] |
DBInputFormat.getSplits(JobConf job,
int chunks)
Logically split the set of input files for the job. |
static void |
DBInputFormat.setInput(JobConf job,
Class<? extends DBWritable> inputClass,
String inputQuery,
String inputCountQuery)
Initializes the map-part of the job with the appropriate input settings. |
static void |
DBInputFormat.setInput(JobConf job,
Class<? extends DBWritable> inputClass,
String tableName,
String conditions,
String orderBy,
String... fieldNames)
Initializes the map-part of the job with the appropriate input settings. |
static void |
DBOutputFormat.setOutput(JobConf job,
String tableName,
int fieldCount)
Initializes the reduce-part of the job with the appropriate output settings |
static void |
DBOutputFormat.setOutput(JobConf job,
String tableName,
String... fieldNames)
Initializes the reduce-part of the job with the appropriate output settings |
Constructors in org.apache.hadoop.mapred.lib.db with parameters of type JobConf | |
---|---|
DBInputFormat.DBRecordReader(DBInputFormat.DBInputSplit split,
Class<T> inputClass,
JobConf job,
Connection conn,
DBConfiguration dbConfig,
String cond,
String[] fields,
String table)
|
Uses of JobConf in org.apache.hadoop.mapred.pipes |
---|
Methods in org.apache.hadoop.mapred.pipes with parameters of type JobConf | |
---|---|
static String |
Submitter.getExecutable(JobConf conf)
Get the URI of the application's executable. |
static boolean |
Submitter.getIsJavaMapper(JobConf conf)
Check whether the job is using a Java Mapper. |
static boolean |
Submitter.getIsJavaRecordReader(JobConf conf)
Check whether the job is using a Java RecordReader |
static boolean |
Submitter.getIsJavaRecordWriter(JobConf conf)
Will the reduce use a Java RecordWriter? |
static boolean |
Submitter.getIsJavaReducer(JobConf conf)
Check whether the job is using a Java Reducer. |
static boolean |
Submitter.getKeepCommandFile(JobConf conf)
Does the user want to keep the command file for debugging? If this is true, pipes will write a copy of the command data to a file in the task directory named "downlink.data", which may be used to run the C++ program under the debugger. |
static RunningJob |
Submitter.jobSubmit(JobConf conf)
Submit a job to the Map-Reduce framework. |
static RunningJob |
Submitter.runJob(JobConf conf)
Submit a job to the map/reduce cluster. |
static void |
Submitter.setExecutable(JobConf conf,
String executable)
Set the URI for the application's executable. |
static void |
Submitter.setIsJavaMapper(JobConf conf,
boolean value)
Set whether the Mapper is written in Java. |
static void |
Submitter.setIsJavaRecordReader(JobConf conf,
boolean value)
Set whether the job is using a Java RecordReader. |
static void |
Submitter.setIsJavaRecordWriter(JobConf conf,
boolean value)
Set whether the job will use a Java RecordWriter. |
static void |
Submitter.setIsJavaReducer(JobConf conf,
boolean value)
Set whether the Reducer is written in Java. |
static void |
Submitter.setKeepCommandFile(JobConf conf,
boolean keep)
Set whether to keep the command file for debugging |
static RunningJob |
Submitter.submitJob(JobConf conf)
Deprecated. Use Submitter.runJob(JobConf) |
Uses of JobConf in org.apache.hadoop.mapreduce.security |
---|
Methods in org.apache.hadoop.mapreduce.security with parameters of type JobConf | |
---|---|
static org.apache.hadoop.security.TokenStorage |
TokenCache.loadTaskTokenStorage(String fileName,
JobConf conf)
load token storage and stores it |
static org.apache.hadoop.security.TokenStorage |
TokenCache.loadTokens(String jobTokenFile,
JobConf conf)
load job token from a file |
Uses of JobConf in org.apache.hadoop.streaming |
---|
Fields in org.apache.hadoop.streaming declared as JobConf | |
---|---|
protected JobConf |
StreamJob.jobConf_
|
Methods in org.apache.hadoop.streaming that return JobConf | |
---|---|
static JobConf |
StreamJob.createJob(String[] argv)
This method creates a streaming job from the given argument list. |
Methods in org.apache.hadoop.streaming with parameters of type JobConf | |
---|---|
void |
AutoInputFormat.configure(JobConf job)
|
void |
PipeMapper.configure(JobConf job)
|
void |
PipeReducer.configure(JobConf job)
|
void |
PipeMapRed.configure(JobConf job)
|
static FileSplit |
StreamUtil.getCurrentSplit(JobConf job)
|
RecordReader |
AutoInputFormat.getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
|
RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> |
StreamInputFormat.getRecordReader(InputSplit genericSplit,
JobConf job,
Reporter reporter)
|
static org.apache.hadoop.streaming.StreamUtil.TaskId |
StreamUtil.getTaskInfo(JobConf job)
|
static boolean |
StreamUtil.isLocalJobTracker(JobConf job)
|
Constructors in org.apache.hadoop.streaming with parameters of type JobConf | |
---|---|
StreamBaseRecordReader(org.apache.hadoop.fs.FSDataInputStream in,
FileSplit split,
Reporter reporter,
JobConf job,
org.apache.hadoop.fs.FileSystem fs)
|
|
StreamXmlRecordReader(org.apache.hadoop.fs.FSDataInputStream in,
FileSplit split,
Reporter reporter,
JobConf job,
org.apache.hadoop.fs.FileSystem fs)
|
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |