org.jwat.warc
Class WarcReaderFactory

java.lang.Object
  org.jwat.warc.WarcReaderFactory

public class WarcReaderFactory
extends Object

Factory used for creating WarcReader instances. The general getReader methods will auto-detect Gzip'ed data and return the appropriate WarcReader instances. The other factory methods can be used to return specific WarcReader instances for compressed or uncompressed records. Readers are available for both sequential and random reading of records. Use of buffered methods and/or buffering speeds up the reader considerably.

Author:

nicl

Field Summary

static int

PUSHBACK_BUFFER_SIZE Buffer size used by PushbackInputStream.

Constructor Summary

protected

WarcReaderFactory() Private constructor to enforce factory method.

Method Summary

static WarcReader	getReader(InputStream in) Creates a new WarcReader from an InputStream.
static WarcReader	getReader(InputStream in, int buffer_size) Creates a new WarcReader from an InputStream wrapped by a BufferedInputStream.
static WarcReader	getReaderCompressed() Creates a new WarcReader without any associated InputStream for random access to GZip compressed records.
static WarcReader	getReaderCompressed(InputStream in) Creates a new WarcReader from an InputStream primarily for random access to GZip compressed records.
static WarcReader	getReaderCompressed(InputStream in, int buffer_size) Creates a new WarcReader from an InputStream wrapped by a BufferedInputStream primarily for random access to GZip compressed records.
static WarcReader	getReaderUncompressed() Creates a new WarcReader without any associated InputStream for random access to uncompressed records.
static WarcReader	getReaderUncompressed(InputStream in) Creates a new WarcReader from an InputStream primarily for random access to uncompressed records.
static WarcReader	getReaderUncompressed(InputStream in, int buffer_size) Creates a new WarcReader from an InputStream wrapped by a BufferedInputStream primarily for random access to uncompressed records.
static boolean	isWarcFile(ByteCountingPushBackInputStream pbin) Check head of PushBackInputStream for a WARC magic number.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PUSHBACK_BUFFER_SIZE

public static final int PUSHBACK_BUFFER_SIZE

Buffer size used by PushbackInputStream.

See Also:

Constant Field Values

Constructor Detail

WarcReaderFactory

protected WarcReaderFactory()

Private constructor to enforce factory method.

Method Detail

isWarcFile

public static boolean isWarcFile(ByteCountingPushBackInputStream pbin)
                          throws IOException

Check head of PushBackInputStream for a WARC magic number.

Parameters:

pbin - PushBackInputStream with WARC records

Returns:

boolean indicating presence of a WARC magic number

Throws:

IOException - if an io error occurs while examining head of stream

getReader

public static WarcReader getReader(InputStream in,
                                   int buffer_size)
                            throws IOException

Creates a new WarcReader from an InputStream wrapped by a BufferedInputStream. The WarcReader implementation returned is chosen based on GZip auto detection.

Parameters:

in - WARC File represented as InputStream

buffer_size - buffer size to use

Returns:

appropriate WarcReader based on data read from InputStream

Throws:

IOException - if an exception occurs during initialization

getReader

public static WarcReader getReader(InputStream in)
                            throws IOException

Creates a new WarcReader from an InputStream. The WarcReader implementation returned is chosen based on GZip auto detection.

Parameters:

in - WARC File represented as InputStream

Returns:

appropriate WarcReader based on data read from InputStream

Throws:

IOException - if an exception occurs during initialization

getReaderUncompressed

public static WarcReader getReaderUncompressed()

Creates a new WarcReader without any associated InputStream for random access to uncompressed records.

Returns:

WarcReader for uncompressed records read from InputStream

getReaderUncompressed

public static WarcReader getReaderUncompressed(InputStream in)
                                        throws IOException

Creates a new WarcReader from an InputStream primarily for random access to uncompressed records.

Parameters:

in - WARC File represented as InputStream

Returns:

WarcReader for uncompressed records read from InputStream

Throws:

IOException - io exception while initializing reader

getReaderUncompressed

public static WarcReader getReaderUncompressed(InputStream in,
                                               int buffer_size)
                                        throws IOException

Creates a new WarcReader from an InputStream wrapped by a BufferedInputStream primarily for random access to uncompressed records.

Parameters:

in - WARC File represented as InputStream

buffer_size - buffer size to use

Returns:

WarcReader for uncompressed records read from InputStream

Throws:

IOException - io exception while initializing reader

getReaderCompressed

public static WarcReader getReaderCompressed()

Creates a new WarcReader without any associated InputStream for random access to GZip compressed records.

Returns:

WarcReader for GZip compressed records read from InputStream

getReaderCompressed

public static WarcReader getReaderCompressed(InputStream in)
                                      throws IOException

Creates a new WarcReader from an InputStream primarily for random access to GZip compressed records.

Parameters:

in - WARC File represented as InputStream

Returns:

WarcReader for GZip compressed records read from InputStream

Throws:

IOException - io exception while initializing reader

getReaderCompressed

public static WarcReader getReaderCompressed(InputStream in,
                                             int buffer_size)
                                      throws IOException

Creates a new WarcReader from an InputStream wrapped by a BufferedInputStream primarily for random access to GZip compressed records.

Parameters:

in - WARC File represented as InputStream

buffer_size - buffer size to use

Returns:

WarcReader for GZip compressed records read from InputStream

Throws:

IOException - io exception while initializing reader