|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.streaming.StreamBaseRecordReader
public abstract class StreamBaseRecordReader
Shared functionality for hadoopStreaming formats. A custom reader can be defined to be a RecordReader with the constructor below and is selected with the option bin/hadoopStreaming -inputreader ...
StreamXmlRecordReader
Field Summary | |
---|---|
protected static org.apache.commons.logging.Log |
LOG
|
Constructor Summary | |
---|---|
StreamBaseRecordReader(org.apache.hadoop.fs.FSDataInputStream in,
FileSplit split,
Reporter reporter,
JobConf job,
org.apache.hadoop.fs.FileSystem fs)
|
Method Summary | |
---|---|
void |
close()
Close this to future operations. |
org.apache.hadoop.io.Text |
createKey()
Create an object of the appropriate type to be used as a key. |
org.apache.hadoop.io.Text |
createValue()
Create an object of the appropriate type to be used as a value. |
long |
getPos()
Returns the current position in the input. |
float |
getProgress()
How much of the input has the RecordReader consumed i.e. |
abstract boolean |
next(org.apache.hadoop.io.Text key,
org.apache.hadoop.io.Text value)
Read a record. |
abstract void |
seekNextRecordBoundary()
Implementation should seek forward in_ to the first byte of the next record. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected static final org.apache.commons.logging.Log LOG
Constructor Detail |
---|
public StreamBaseRecordReader(org.apache.hadoop.fs.FSDataInputStream in, FileSplit split, Reporter reporter, JobConf job, org.apache.hadoop.fs.FileSystem fs) throws IOException
IOException
Method Detail |
---|
public abstract boolean next(org.apache.hadoop.io.Text key, org.apache.hadoop.io.Text value) throws IOException
next
in interface RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
key
- the key to read data intovalue
- the value to read data into
IOException
public long getPos() throws IOException
getPos
in interface RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
IOException
public void close() throws IOException
close
in interface RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
IOException
public float getProgress() throws IOException
RecordReader
RecordReader
consumed i.e.
has been processed by?
getProgress
in interface RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
0.0
to 1.0
.
IOException
public org.apache.hadoop.io.Text createKey()
RecordReader
createKey
in interface RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
public org.apache.hadoop.io.Text createValue()
RecordReader
createValue
in interface RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>
public abstract void seekNextRecordBoundary() throws IOException
IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |