A B C D E F G H I L M O P R S T W

A

addEmptyWarning(String) - Method in class org.jwat.warc.WarcFieldParsers

Add a warning diagnosis on the given entity stating that it is empty.

addErrorDiagnosis(DiagnosisType, String, String...) - Method in class org.jwat.warc.WarcHeader

Add an error diagnosis of the given type on a specific entity with optional extra information.

addErrorDiagnosis(DiagnosisType, String, String...) - Method in class org.jwat.warc.WarcRecord

Add an error diagnosis of the given type on a specific entity with optional extra information.

addHeader(HeaderLine) - Method in class org.jwat.warc.WarcHeader

Identify a (WARC) header name, validate the value and set the header.

addHeader(String, String) - Method in class org.jwat.warc.WarcHeader

addHeader(String, Integer, String) - Method in class org.jwat.warc.WarcHeader

addHeader(String, Long, String) - Method in class org.jwat.warc.WarcHeader

addHeader(String, WarcDigest, String) - Method in class org.jwat.warc.WarcHeader

addHeader(String, ContentType, String) - Method in class org.jwat.warc.WarcHeader

addHeader(String, Date, String) - Method in class org.jwat.warc.WarcHeader

addHeader(String, InetAddress, String) - Method in class org.jwat.warc.WarcHeader

addHeader(String, URI, String) - Method in class org.jwat.warc.WarcHeader

addHeader(String, String, int, Integer, Long, WarcDigest, ContentType, Date, InetAddress, URI) - Method in class org.jwat.warc.WarcHeader

addInvalidExpectedError(String, String...) - Method in class org.jwat.warc.WarcFieldParsers

Add an error diagnosis on the given entity stating that it is invalid and something else was expected.

addWarningDiagnosis(DiagnosisType, String, String...) - Method in class org.jwat.warc.WarcHeader

Add a warning diagnosis of the given type on a specific entity with optional extra information.

addWarningDiagnosis(DiagnosisType, String, String...) - Method in class org.jwat.warc.WarcRecord

Add a warning diagnosis of the given type on a specific entity with optional extra information.

B

bBlockDigest - Variable in class org.jwat.warc.WarcReader

Block Digest enabled/disabled.

bClosed - Variable in class org.jwat.warc.WarcRecord

Has record been closed before.

bExceptionOnContentLengthMismatch - Variable in class org.jwat.warc.WarcWriter

Configuration for throwing exception on content-length mismatch.

bIsCompliant - Variable in class org.jwat.warc.WarcReader

Compliance status for records parsed up to now.

bIsCompliant - Variable in class org.jwat.warc.WarcRecord

Is this record compliant ie. error free.

blockDigestAlgorithm - Variable in class org.jwat.warc.WarcReader

Default block digest algorithm to use if none is present in the record.

blockDigestEncoding - Variable in class org.jwat.warc.WarcReader

Default encoding scheme used to encode block digest into a string, if none is detected from the record.

bMisplacedCr - Variable in class org.jwat.warc.WarcRecord

Did the reader detect a misplaced CR while parsing newlines.

bMisplacedLf - Variable in class org.jwat.warc.WarcRecord

Did the reader detect a misplaced LF while parsing newlines.

bMissingCr - Variable in class org.jwat.warc.WarcRecord

Did the reader detect a missing CR while parsing newlines.

bMissingLf - Variable in class org.jwat.warc.WarcRecord

Did the reader detect a missing LF while parsing newlines.

bPayloadClosed - Variable in class org.jwat.warc.WarcRecord

Has payload been closed before.

bPayloadDigest - Variable in class org.jwat.warc.WarcReader

Payload Digest enabled/disabled.

bufferSize - Variable in class org.jwat.warc.WarcReaderCompressed

Buffer size, if any, to use on GZip entry InputStream.

C

checkFieldPolicy(int, int, Object, String) - Method in class org.jwat.warc.WarcHeader

Given a WARC record type and a WARC field looks up the policy in a matrix build from the WARC ISO standard.

checkFields() - Method in class org.jwat.warc.WarcHeader

Validate the WARC header relative to the WARC-Type and according the WARC ISO standard.

close() - Method in class org.jwat.warc.WarcReader

Close current record resource(s) and input stream(s).

close() - Method in class org.jwat.warc.WarcReaderCompressed

close() - Method in class org.jwat.warc.WarcReaderUncompressed

close() - Method in class org.jwat.warc.WarcRecord

Close resources associated with the WARC record.

close() - Method in class org.jwat.warc.WarcWriter

Close WARC writer and free its resources.

close() - Method in class org.jwat.warc.WarcWriterCompressed

close() - Method in class org.jwat.warc.WarcWriterUncompressed

closeRecord() - Method in class org.jwat.warc.WarcWriter

Close the WARC record.

closeRecord() - Method in class org.jwat.warc.WarcWriterCompressed

closeRecord() - Method in class org.jwat.warc.WarcWriterUncompressed

closeRecord_impl() - Method in class org.jwat.warc.WarcWriter

Close the WARC record by writing two newlines.

computedBlockDigest - Variable in class org.jwat.warc.WarcRecord

Computed block digest.

computedPayloadDigest - Variable in class org.jwat.warc.WarcRecord

Computed payload digest.

consumed - Variable in class org.jwat.warc.WarcReader

Number of bytes consumed by this reader.

CONTENT_TYPE_FORMAT - Static variable in class org.jwat.warc.WarcConstants

Content-type format string as specified in RFC2616.

CONTENT_TYPE_METADATA - Static variable in class org.jwat.warc.WarcConstants

Suggested content-type for metadata records and others.

contentLength - Variable in class org.jwat.warc.WarcHeader

contentLengthStr - Variable in class org.jwat.warc.WarcHeader

contentType - Variable in class org.jwat.warc.WarcHeader

contentTypeStr - Variable in class org.jwat.warc.WarcHeader

createRecord(WarcWriter) - Static method in class org.jwat.warc.WarcRecord

createWarcDigest(String, byte[], String, String) - Static method in class org.jwat.warc.WarcDigest

Create an object with the supplied parameters.

CT_APP_WARC_FIELDS - Static variable in class org.jwat.warc.WarcConstants

Suggested content-type/media-type for metadata records and others.

D

diagnostics - Variable in class org.jwat.warc.WarcFieldParsers

Diagnostics used to report diagnoses.

diagnostics - Variable in class org.jwat.warc.WarcHeader

Diagnostics used to report diagnoses.

diagnostics - Variable in class org.jwat.warc.WarcReader

Reader level errors and warnings or when no record is available.

diagnostics - Variable in class org.jwat.warc.WarcRecord

Validation errors and warnings.

diagnostics - Variable in class org.jwat.warc.WarcWriter

Writer level errors and warnings or when writing byte headers.

E

endMark - Static variable in class org.jwat.warc.WarcConstants

End mark used after each record consisting of two newlines.

entry - Variable in class org.jwat.warc.WarcWriterCompressed

Current GZip entry.

equals(Object) - Method in class org.jwat.warc.WarcConcurrentTo

errors - Variable in class org.jwat.warc.WarcReader

Aggregated number of errors encountered while parsing.

exceptionOnContentLengthMismatch() - Method in class org.jwat.warc.WarcWriter

Does this writer throw an exception if the content-length does not match the payload amount written.

F

FDT_CONTENTTYPE - Static variable in class org.jwat.warc.WarcConstants

WARC ContentType field datatype identifier.

FDT_DATE - Static variable in class org.jwat.warc.WarcConstants

WARC Date field datatype identifier.

FDT_DIGEST - Static variable in class org.jwat.warc.WarcConstants

WARC Digest field datatype identifier.

FDT_IDX_STRINGS - Static variable in class org.jwat.warc.WarcConstants

WARC field datatype id to field datatype name mapping table.

FDT_INETADDRESS - Static variable in class org.jwat.warc.WarcConstants

WARC InetAddress field datatype identifier.

FDT_INTEGER - Static variable in class org.jwat.warc.WarcConstants

WARC Integer field datatype identifier.

FDT_LONG - Static variable in class org.jwat.warc.WarcConstants

WARC Long field datatype identifier.

FDT_STRING - Static variable in class org.jwat.warc.WarcConstants

WARC String field datatype identifier.

FDT_URI - Static variable in class org.jwat.warc.WarcConstants

WARC URI field datatype identifier.

field_policy - Static variable in class org.jwat.warc.WarcConstants

A (Warc-Types x Warc-Header-Fields) matrix used for policy validation.

fieldNameIdxMap - Static variable in class org.jwat.warc.WarcConstants

Map used to identify known warc field names.

fieldNamesRepeatableLookup - Static variable in class org.jwat.warc.WarcConstants

Lookup table of Warc fields that can have multiple occurrences.

fieldParser - Variable in class org.jwat.warc.WarcHeader

WARC field parser used.

fieldParser - Variable in class org.jwat.warc.WarcReader

WARC field parser used.

fieldParser - Variable in class org.jwat.warc.WarcWriter

WARC field parser used.

FN_CONTENT_LENGTH - Static variable in class org.jwat.warc.WarcConstants

Content-length field name.

FN_CONTENT_TYPE - Static variable in class org.jwat.warc.WarcConstants

Content-type field name.

FN_IDX_CONTENT_LENGTH - Static variable in class org.jwat.warc.WarcConstants

Warc reader content-length field name id.

FN_IDX_CONTENT_TYPE - Static variable in class org.jwat.warc.WarcConstants

Warc reader content-type field name id.

FN_IDX_DT - Static variable in class org.jwat.warc.WarcConstants

Array to lookup WARC field datatypes.

FN_IDX_STRINGS - Static variable in class org.jwat.warc.WarcConstants

WARC field name id to field name mapping table.

FN_IDX_WARC_BLOCK_DIGEST - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-block-digest field name id.

FN_IDX_WARC_CONCURRENT_TO - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-concurrent-to field name id.

FN_IDX_WARC_DATE - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-date field name id.

FN_IDX_WARC_FILENAME - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-filename field name id.

FN_IDX_WARC_IDENTIFIED_PAYLOAD_TYPE - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-identified-payload-type field name id.

FN_IDX_WARC_IP_ADDRESS - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-ip-address field name id.

FN_IDX_WARC_PAYLOAD_DIGEST - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-payload-digest field name id.

FN_IDX_WARC_PROFILE - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-profile field name id.

FN_IDX_WARC_RECORD_ID - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-record-id field name id.

FN_IDX_WARC_REFERS_TO - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-refers-to field name id.

FN_IDX_WARC_SEGMENT_NUMBER - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-segment-number field name id.

FN_IDX_WARC_SEGMENT_ORIGIN_ID - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-segment-origin-id field name id.

FN_IDX_WARC_SEGMENT_TOTAL_LENGTH - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-segment-totalt-length field name id.

FN_IDX_WARC_TARGET_URI - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-target-uri field name id.

FN_IDX_WARC_TRUNCATED - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-truncated field name id.

FN_IDX_WARC_TYPE - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-type field name id.

FN_IDX_WARC_WARCINFO_ID - Static variable in class org.jwat.warc.WarcConstants

Warc reader warc-warcinfo-id field name id.

FN_MAX_NUMBER - Static variable in class org.jwat.warc.WarcConstants

Number of WARC fields (zero-indexed).

FN_WARC_BLOCK_DIGEST - Static variable in class org.jwat.warc.WarcConstants

Warc-block-digest field name.

FN_WARC_CONCURRENT_TO - Static variable in class org.jwat.warc.WarcConstants

Warc-concurrent-to field name.

FN_WARC_DATE - Static variable in class org.jwat.warc.WarcConstants

Warc-date field name.

FN_WARC_FILENAME - Static variable in class org.jwat.warc.WarcConstants

Warc-filename field name.

FN_WARC_IDENTIFIED_PAYLOAD_TYPE - Static variable in class org.jwat.warc.WarcConstants

Warc-identified-payload-type field name.

FN_WARC_IP_ADDRESS - Static variable in class org.jwat.warc.WarcConstants

Warc-ip-address field name.

FN_WARC_PAYLOAD_DIGEST - Static variable in class org.jwat.warc.WarcConstants

Warc-payload-digest field name.

FN_WARC_PROFILE - Static variable in class org.jwat.warc.WarcConstants

Warc-profile field name.

FN_WARC_RECORD_ID - Static variable in class org.jwat.warc.WarcConstants

Warc-record-id field name.

FN_WARC_REFERS_TO - Static variable in class org.jwat.warc.WarcConstants

Warc-refers-to field name.

FN_WARC_SEGMENT_NUMBER - Static variable in class org.jwat.warc.WarcConstants

Warc-segment-number field name.

FN_WARC_SEGMENT_ORIGIN_ID - Static variable in class org.jwat.warc.WarcConstants

Warc-segment-origin-id field name.

FN_WARC_SEGMENT_TOTAL_LENGTH - Static variable in class org.jwat.warc.WarcConstants

Warc-segment-totalt-length field name.

FN_WARC_TARGET_URI - Static variable in class org.jwat.warc.WarcConstants

Warc-target-uri field name.

FN_WARC_TRUNCATED - Static variable in class org.jwat.warc.WarcConstants

Warc-truncated field name.

FN_WARC_TYPE - Static variable in class org.jwat.warc.WarcConstants

Warc-type field name.

FN_WARC_WARCINFO_ID - Static variable in class org.jwat.warc.WarcConstants

Warc-warcinfo-id field name.

G

getBlockDigestAlgorithm() - Method in class org.jwat.warc.WarcReader

Get the default block digest algorithm.

getBlockDigestEnabled() - Method in class org.jwat.warc.WarcReader

Get the readers block digest on/off status.

getBlockDigestEncoding() - Method in class org.jwat.warc.WarcReader

Get the default block digest encoding scheme.

getConsumed() - Method in class org.jwat.warc.WarcReader

Get number of bytes consumed by this reader.

getConsumed() - Method in class org.jwat.warc.WarcReaderCompressed

getConsumed() - Method in class org.jwat.warc.WarcReaderUncompressed

getDate(String) - Static method in class org.jwat.warc.WarcDateParser

Parses the date using the format "yyyy-MM-ddTHH:mm:ssZ".

getDateFormat() - Static method in class org.jwat.warc.WarcDateParser

Return a DateFormat object which can be used to string format WARC dates.

getHeader(String) - Method in class org.jwat.warc.WarcRecord

Get a non-standard WARC header or null, if nothing is stored for this header name.

getHeaderList() - Method in class org.jwat.warc.WarcRecord

Get a List of all the non-standard WARC headers found during parsing.

getIteratorExceptionThrown() - Method in class org.jwat.warc.WarcReader

Gets an exception thrown in the iterator if any or null.

getNextRecord() - Method in class org.jwat.warc.WarcReader

Parses and gets the next record.

getNextRecord() - Method in class org.jwat.warc.WarcReaderCompressed

getNextRecord() - Method in class org.jwat.warc.WarcReaderUncompressed

getNextRecordFrom(InputStream, long) - Method in class org.jwat.warc.WarcReader

Parses and gets the next record from an Inputstream.

getNextRecordFrom(InputStream, long, int) - Method in class org.jwat.warc.WarcReader

Parses and gets the next record from an Inputstream wrapped by a BufferedInputStream.

getNextRecordFrom(InputStream, long) - Method in class org.jwat.warc.WarcReaderCompressed

getNextRecordFrom(InputStream, long, int) - Method in class org.jwat.warc.WarcReaderCompressed

getNextRecordFrom(InputStream, long) - Method in class org.jwat.warc.WarcReaderUncompressed

getNextRecordFrom(InputStream, long, int) - Method in class org.jwat.warc.WarcReaderUncompressed

getOffset() - Method in class org.jwat.warc.WarcReader

Get the current offset in the WARC InputStream.

getOffset() - Method in class org.jwat.warc.WarcReaderCompressed

Get the current offset in the WARC GzipReader.

getOffset() - Method in class org.jwat.warc.WarcReaderUncompressed

getPayload() - Method in class org.jwat.warc.WarcRecord

Return Payload object.

getPayloadContent() - Method in class org.jwat.warc.WarcRecord

Payload content InputStream getter.

getPayloadDigestAlgorithm() - Method in class org.jwat.warc.WarcReader

Get the default payload digest algorithm.

getPayloadDigestEnabled() - Method in class org.jwat.warc.WarcReader

Get the readers payload digest on/off status.

getPayloadDigestEncoding() - Method in class org.jwat.warc.WarcReader

Get the default payload digest encoding scheme.

getPayloadHeaderMaxSize() - Method in class org.jwat.warc.WarcReader

Get the max size allowed for a payload header.

getReader(InputStream, int) - Static method in class org.jwat.warc.WarcReaderFactory

Creates a new WarcReader from an InputStream wrapped by a BufferedInputStream.

getReader(InputStream) - Static method in class org.jwat.warc.WarcReaderFactory

Creates a new WarcReader from an InputStream.

getReaderCompressed() - Static method in class org.jwat.warc.WarcReaderFactory

Creates a new WarcReader without any associated InputStream for random access to GZip compressed records.

getReaderCompressed(InputStream) - Static method in class org.jwat.warc.WarcReaderFactory

Creates a new WarcReader from an InputStream primarily for random access to GZip compressed records.

getReaderCompressed(InputStream, int) - Static method in class org.jwat.warc.WarcReaderFactory

Creates a new WarcReader from an InputStream wrapped by a BufferedInputStream primarily for random access to GZip compressed records.

getReaderUncompressed() - Static method in class org.jwat.warc.WarcReaderFactory

Creates a new WarcReader without any associated InputStream for random access to uncompressed records.

getReaderUncompressed(InputStream) - Static method in class org.jwat.warc.WarcReaderFactory

Creates a new WarcReader from an InputStream primarily for random access to uncompressed records.

getReaderUncompressed(InputStream, int) - Static method in class org.jwat.warc.WarcReaderFactory

Creates a new WarcReader from an InputStream wrapped by a BufferedInputStream primarily for random access to uncompressed records.

getStartOffset() - Method in class org.jwat.warc.WarcReader

Get the offset of the current or next WARC record.

getStartOffset() - Method in class org.jwat.warc.WarcReaderCompressed

Get the current offset in the WARC GzipReader.

getStartOffset() - Method in class org.jwat.warc.WarcReaderUncompressed

getStartOffset() - Method in class org.jwat.warc.WarcRecord

Get the record offset relative to the start of the WARC file InputStream.

getWarcHeaderMaxSize() - Method in class org.jwat.warc.WarcReader

Get the max size allowed for a WARC header.

getWriter(OutputStream, boolean) - Static method in class org.jwat.warc.WarcWriterFactory

Creates a new unbuffered WarcWriter from an OutputStream.

getWriter(OutputStream, int, boolean) - Static method in class org.jwat.warc.WarcWriterFactory

Creates a new buffered WarcWriter from an OutputStream.

getWriterCompressed(OutputStream) - Static method in class org.jwat.warc.WarcWriterFactory

Creates a new unbuffered compressing WarcWriter from an OutputStream.

getWriterCompressed(OutputStream, int) - Static method in class org.jwat.warc.WarcWriterFactory

Creates a new buffered compressing WarcWriter from an OutputStream.

getWriterUncompressed(OutputStream) - Static method in class org.jwat.warc.WarcWriterFactory

Creates a new unbuffered normal WarcWriter from an OutputStream.

getWriterUncompressed(OutputStream, int) - Static method in class org.jwat.warc.WarcWriterFactory

Creates a new buffered normal WarcWriter from an OutputStream.

H

hashCode() - Method in class org.jwat.warc.WarcConcurrentTo

hasPayload() - Method in class org.jwat.warc.WarcRecord

Specifies whether this record has a payload or not.

header - Variable in class org.jwat.warc.WarcRecord

WARC header.

header - Variable in class org.jwat.warc.WarcWriter

Current WARC header written.

headerBytes - Variable in class org.jwat.warc.WarcHeader

Raw WARC header byte array.

headerBytesOut - Variable in class org.jwat.warc.WarcHeader

Raw WARC header output stream.

headerContentLength - Variable in class org.jwat.warc.WarcWriter

Content-Length from the WARC header.

headerLineReader - Variable in class org.jwat.warc.WarcReader

Header line reader used to read the WARC headers.

headerList - Variable in class org.jwat.warc.WarcHeader

List of parsed header fields.

headerMap - Variable in class org.jwat.warc.WarcHeader

Map of parsed header fields.

httpHeader - Variable in class org.jwat.warc.WarcRecord

HTTP header content parsed from payload.

I

in - Variable in class org.jwat.warc.WarcReaderUncompressed

WARC file InputStream.

in - Variable in class org.jwat.warc.WarcRecord

Input stream used to read this record.

init() - Method in class org.jwat.warc.WarcReader

Method used to initialize a readers internal state.

init() - Method in class org.jwat.warc.WarcWriter

Method used to initialize a readers internal state.

initHeader(WarcWriter, Diagnostics<Diagnosis>) - Static method in class org.jwat.warc.WarcHeader

initHeader(WarcReader, long, Diagnostics<Diagnosis>) - Static method in class org.jwat.warc.WarcHeader

isClosed() - Method in class org.jwat.warc.WarcRecord

Check to see if the record has been closed.

isCompliant() - Method in class org.jwat.warc.WarcReader

Returns a boolean indicating if all records parsed so far are compliant.

isCompliant() - Method in class org.jwat.warc.WarcRecord

Returns a boolean indicating the ISO compliance status of this record.

isCompressed() - Method in class org.jwat.warc.WarcReader

Is this reader assuming GZip compressed input.

isCompressed() - Method in class org.jwat.warc.WarcReaderCompressed

isCompressed() - Method in class org.jwat.warc.WarcReaderUncompressed

isCompressed() - Method in class org.jwat.warc.WarcWriter

Is this writer producing compressed output.

isCompressed() - Method in class org.jwat.warc.WarcWriterCompressed

isCompressed() - Method in class org.jwat.warc.WarcWriterUncompressed

isValidBlockDigest - Variable in class org.jwat.warc.WarcRecord

Is Warc-Block-Digest valid.

isValidPayloadDigest - Variable in class org.jwat.warc.WarcRecord

Is Warc-Payload-Digest valid.

isWarcFile(ByteCountingPushBackInputStream) - Static method in class org.jwat.warc.WarcReaderFactory

Check head of PushBackInputStream for a WARC magic number.

iterator() - Method in class org.jwat.warc.WarcReader

Returns an Iterator over the records as they are being parsed.

iteratorExceptionThrown - Variable in class org.jwat.warc.WarcReader

Exception thrown while using the iterator.

L

lineReader - Variable in class org.jwat.warc.WarcReader

Line reader used to read version lines.

M

MEDIA_TYPE_METADATA - Static variable in class org.jwat.warc.WarcConstants

Suggested media-type for metadata records and others.

O

org.jwat.warc - package org.jwat.warc

out - Variable in class org.jwat.warc.WarcWriter

Outputstream used to write WARC records.

P

P_IDX_STRINGS - Static variable in class org.jwat.warc.WarcConstants

WARC profile id to field name mapping table.

parseContentType(String, String) - Method in class org.jwat.warc.WarcFieldParsers

Parse and validate content-type string with optional parameters.

parseDate(String, String) - Method in class org.jwat.warc.WarcFieldParsers

Parses WARC record date.

parseDigest(String, String) - Method in class org.jwat.warc.WarcFieldParsers

Parse and validate WARC digest string.

parseHeader(ByteCountingPushBackInputStream) - Method in class org.jwat.warc.WarcHeader

parseHeaders(ByteCountingPushBackInputStream) - Method in class org.jwat.warc.WarcHeader

Reads WARC header lines one line at a time until an empty line is encountered.

parseInteger(String, String) - Method in class org.jwat.warc.WarcFieldParsers

Returns an Integer object holding the value of the specified string.

parseIpAddress(String, String) - Method in class org.jwat.warc.WarcFieldParsers

Parses WARC record IP address.

parseLong(String, String) - Method in class org.jwat.warc.WarcFieldParsers

Returns a Long object holding the value of the specified string.

parseNewLines(ByteCountingPushBackInputStream) - Method in class org.jwat.warc.WarcRecord

Looks forward in the inputstream and counts the number of newlines found.

parseRecord(ByteCountingPushBackInputStream, WarcReader) - Static method in class org.jwat.warc.WarcRecord

Given an InputStream it tries to read and validate a WARC header block.

parseString(String, String) - Method in class org.jwat.warc.WarcFieldParsers

Parses a string.

parseUri(String, String) - Method in class org.jwat.warc.WarcFieldParsers

Returns an URL object holding the value of the specified string.

parseVersion(ByteCountingPushBackInputStream) - Method in class org.jwat.warc.WarcHeader

Looks forward in the inputstream for a valid WARC version line.

parseWarcDigest(String) - Static method in class org.jwat.warc.WarcDigest

Parse and validate the format of a WARC digest header value.

payload - Variable in class org.jwat.warc.WarcRecord

Payload object if any exists.

payloadClosed() - Method in class org.jwat.warc.WarcRecord

Called when the payload object is closed and final steps in the validation process can be performed.

payloadDigestAlgorithm - Variable in class org.jwat.warc.WarcReader

Default payload digest algorithm to use if none is present in the record.

payloadDigestEncoding - Variable in class org.jwat.warc.WarcReader

Default encoding scheme used to encode payload digest into a string, if none is detected from the record.

payloadHeaderMaxSize - Variable in class org.jwat.warc.WarcReader

Max size allowed for a payload header.

payloadWrittenTotal - Variable in class org.jwat.warc.WarcWriter

Total bytes written for current record payload.

POLICY_IGNORE - Static variable in class org.jwat.warc.WarcConstants

Warc header can be ignored.

POLICY_MANDATORY - Static variable in class org.jwat.warc.WarcConstants

Warc header is mandatory (equal to shall).

POLICY_MAY - Static variable in class org.jwat.warc.WarcConstants

Warc header can be present.

POLICY_MAY_NOT - Static variable in class org.jwat.warc.WarcConstants

Warc header should not be present.

POLICY_SHALL - Static variable in class org.jwat.warc.WarcConstants

Warc header must be present.

POLICY_SHALL_NOT - Static variable in class org.jwat.warc.WarcConstants

Warc header must not be present.

processComputedDigest(WarcDigest, String, String, String) - Method in class org.jwat.warc.WarcRecord

Adjust algorithm and encoding information about computed block digest.

processWarcDigest(WarcDigest, WarcDigest, String) - Method in class org.jwat.warc.WarcRecord

Auto-detect encoding used in WARC digest header and compare it to the internal one, if it has been computed.

PROFILE_IDENTICAL_PAYLOAD_DIGEST - Static variable in class org.jwat.warc.WarcConstants

Revisit WARC-Profile id for identical payload digest.

PROFILE_IDX_IDENTICAL_PAYLOAD_DIGEST - Static variable in class org.jwat.warc.WarcConstants

Warc reader id for identical payload digest profile.

PROFILE_IDX_SERVER_NOT_MODIFIED - Static variable in class org.jwat.warc.WarcConstants

Warc reader id for server not modified profile.

PROFILE_IDX_UNKNOWN - Static variable in class org.jwat.warc.WarcConstants

Warc reader id for unknown profile.

PROFILE_SERVER_NOT_MODIFIED - Static variable in class org.jwat.warc.WarcConstants

Revisit WARC-Profile id for server not modified.

profileIdxMap - Static variable in class org.jwat.warc.WarcConstants

Profile lookup map used to identify WARC-Profile values.

PUSHBACK_BUFFER_SIZE - Static variable in class org.jwat.warc.WarcReaderCompressed

Buffer size used by PushbackInputStream.

PUSHBACK_BUFFER_SIZE - Static variable in class org.jwat.warc.WarcReaderFactory

Buffer size used by PushbackInputStream.

PUSHBACK_BUFFER_SIZE - Static variable in class org.jwat.warc.WarcReaderUncompressed

Buffer size used by PushbackInputStream.

R

reader - Variable in class org.jwat.warc.WarcHeader

Associated WarcReader context.

reader - Variable in class org.jwat.warc.WarcReaderCompressed

WARC file InputStream.

reader - Variable in class org.jwat.warc.WarcRecord

Reader instance used, required for file compliance.

recordTypeIdxMap - Static variable in class org.jwat.warc.WarcConstants

WARC-Type lookup map.

RT_CONTINUATION - Static variable in class org.jwat.warc.WarcConstants

WARC-Type continuation id.

RT_CONVERSION - Static variable in class org.jwat.warc.WarcConstants

WARC-Type conversion id.

RT_IDX_CONTINUATION - Static variable in class org.jwat.warc.WarcConstants

Warc reader continuation warc record type id.

RT_IDX_CONVERSION - Static variable in class org.jwat.warc.WarcConstants

Warc reader conversion warc record type id.

RT_IDX_METADATA - Static variable in class org.jwat.warc.WarcConstants

Warc reader metadata warc record type id.

RT_IDX_REQUEST - Static variable in class org.jwat.warc.WarcConstants

Warc reader request warc record type id.

RT_IDX_RESOURCE - Static variable in class org.jwat.warc.WarcConstants

Warc reader resource warc record type id.

RT_IDX_RESPONSE - Static variable in class org.jwat.warc.WarcConstants

Warc reader response warc record type id.

RT_IDX_REVISIT - Static variable in class org.jwat.warc.WarcConstants

Warc reader revisit warc record type id.

RT_IDX_STRINGS - Static variable in class org.jwat.warc.WarcConstants

WARC type id to field name mapping table.

RT_IDX_UNKNOWN - Static variable in class org.jwat.warc.WarcConstants

Warc reader unknown warc record type id.

RT_IDX_WARCINFO - Static variable in class org.jwat.warc.WarcConstants

Warc reader warcinfo warc record type id.

RT_MAX_NUMBER - Static variable in class org.jwat.warc.WarcConstants

Number of WARC types (zero indexed).

RT_METADATA - Static variable in class org.jwat.warc.WarcConstants

WARC-Type metadata id.

RT_REQUEST - Static variable in class org.jwat.warc.WarcConstants

WARC-Type request id.

RT_RESOURCE - Static variable in class org.jwat.warc.WarcConstants

WARC-Type resource id.

RT_RESPONSE - Static variable in class org.jwat.warc.WarcConstants

WARC-Type response id.

RT_REVISIT - Static variable in class org.jwat.warc.WarcConstants

WARC-Type revisit id.

RT_WARCINFO - Static variable in class org.jwat.warc.WarcConstants

WARC-Type warcinfo id.

S

S_HEADER_WRITTEN - Static variable in class org.jwat.warc.WarcWriter

State after header has been written.

S_INIT - Static variable in class org.jwat.warc.WarcWriter

State after writer has been constructed and before records have been written.

S_PAYLOAD_WRITTEN - Static variable in class org.jwat.warc.WarcWriter

State after payload has been written.

S_RECORD_CLOSED - Static variable in class org.jwat.warc.WarcWriter

State after record has been closed.

seen - Variable in class org.jwat.warc.WarcHeader

setBlockDigestAlgorithm(String) - Method in class org.jwat.warc.WarcReader

Set the default block digest algorithm.

setBlockDigestEnabled(boolean) - Method in class org.jwat.warc.WarcReader

Set the readers block digest on/off status.

setBlockDigestEncoding(String) - Method in class org.jwat.warc.WarcReader

Set the default block digest encoding scheme.

setExceptionOnContentLengthMismatch(boolean) - Method in class org.jwat.warc.WarcWriter

Tell the writer what to do in case of mismatch between content-length and amount payload written.

setPayloadDigestAlgorithm(String) - Method in class org.jwat.warc.WarcReader

Set the default payload digest algorithm.

setPayloadDigestEnabled(boolean) - Method in class org.jwat.warc.WarcReader

Set the readers payload digest on/off status.

setPayloadDigestEncoding(String) - Method in class org.jwat.warc.WarcReader

Set the default payload digest encoding scheme.

setPayloadHeaderMaxSize(int) - Method in class org.jwat.warc.WarcReader

Set the max size allowed for a payload header.

setWarcHeaderMaxSize(int) - Method in class org.jwat.warc.WarcReader

Set the max size allowed for a WARC header.

startOffset - Variable in class org.jwat.warc.WarcHeader

startOffset - Variable in class org.jwat.warc.WarcReaderCompressed

Cached start offset used after the reader is closed.

startOffset - Variable in class org.jwat.warc.WarcReaderUncompressed

Start offset of current or next valid record.

state - Variable in class org.jwat.warc.WarcWriter

Current state of writer.

stream_copy_buffer - Variable in class org.jwat.warc.WarcWriter

Buffer used by streamPayload() to copy from one stream to another.

streamPayload(InputStream) - Method in class org.jwat.warc.WarcWriter

streamPayload(InputStream) - Method in class org.jwat.warc.WarcWriterCompressed

T

toString() - Method in class org.jwat.warc.WarcDigest

Returns a header representation of the class state.

toStringFull() - Method in class org.jwat.warc.WarcDigest

truncatedTypeIdxMap - Static variable in class org.jwat.warc.WarcConstants

Lookup map for known truncation reason id's.

TT_DISCONNECT - Static variable in class org.jwat.warc.WarcConstants

WARC-Truncated disconnect id.

TT_IDX_DISCONNECT - Static variable in class org.jwat.warc.WarcConstants

Warc reader disconnect reason id.

TT_IDX_FUTURE_REASON - Static variable in class org.jwat.warc.WarcConstants

Warc reader future reason id.

TT_IDX_LENGTH - Static variable in class org.jwat.warc.WarcConstants

Warc reader length reason id.

TT_IDX_STRINGS - Static variable in class org.jwat.warc.WarcConstants

WARC truncation reason id to field name mapping table.

TT_IDX_TIME - Static variable in class org.jwat.warc.WarcConstants

Warc reader time reason id.

TT_IDX_UNSPECIFIED - Static variable in class org.jwat.warc.WarcConstants

Warc reader unspecified reason id.

TT_LENGTH - Static variable in class org.jwat.warc.WarcConstants

WARC-Truncated length id.

TT_TIME - Static variable in class org.jwat.warc.WarcConstants

WARC-Truncated time id

TT_UNSPECIFIED - Static variable in class org.jwat.warc.WarcConstants

WARC-Truncated unspecified id.

W

WARC_DATE_FORMAT - Static variable in class org.jwat.warc.WarcConstants

WARC date format string as specified by the WARC ISO standard.

WARC_DIGEST_FORMAT - Static variable in class org.jwat.warc.WarcConstants

WARC digest format string as specified by the WARC ISO standard.

WARC_MAGIC_HEADER - Static variable in class org.jwat.warc.WarcConstants

A WARC header block starts with this string including trailing version information.

WARC_MIME_TYPE - Static variable in class org.jwat.warc.WarcConstants

WARC mime type.

WARC_RECORD_TRAILING_NEWLINES - Static variable in class org.jwat.warc.WarcConstants

Trailing newlines after each record as per the WARC ISO standard.

warcBlockDigest - Variable in class org.jwat.warc.WarcHeader

warcBlockDigestStr - Variable in class org.jwat.warc.WarcHeader

WarcConcurrentTo - Class in org.jwat.warc

Simple wrapper for a (non) validated WARC ConcurrentTo header.

WarcConcurrentTo() - Constructor for class org.jwat.warc.WarcConcurrentTo

warcConcurrentToList - Variable in class org.jwat.warc.WarcHeader

warcConcurrentToStr - Variable in class org.jwat.warc.WarcConcurrentTo

Warc-Concurrent-To string representation.

warcConcurrentToUri - Variable in class org.jwat.warc.WarcConcurrentTo

Warc-Concurrent-To URI object.

WarcConstants - Class in org.jwat.warc

Class containing all relevant WARC constants and structures.

WarcConstants() - Constructor for class org.jwat.warc.WarcConstants

This utility class does not require instantiation.

warcDate - Variable in class org.jwat.warc.WarcHeader

warcDateFormat - Variable in class org.jwat.warc.WarcHeader

WARC DateFormat as specified by the WARC ISO standard.

warcDateFormat - Variable in class org.jwat.warc.WarcWriter

WARC DateFormat as specified by the WARC ISO standard.

WarcDateParser - Class in org.jwat.warc

WARC-Date parser and format validator.

warcDateStr - Variable in class org.jwat.warc.WarcHeader

WarcDigest - Class in org.jwat.warc

This class represents the parsed and format validated information provided from a WARC digest header value.

WarcDigest() - Constructor for class org.jwat.warc.WarcDigest

Package level constructor.

WarcDigest(String, String) - Constructor for class org.jwat.warc.WarcDigest

Construct an object with the supplied parameters.

WarcFieldParsers - Class in org.jwat.warc

Separate class containing all the different types of field parser.

WarcFieldParsers() - Constructor for class org.jwat.warc.WarcFieldParsers

warcFilename - Variable in class org.jwat.warc.WarcHeader

WarcHeader - Class in org.jwat.warc

WarcHeader() - Constructor for class org.jwat.warc.WarcHeader

Non public constructor to allow unit testing.

warcHeaderMaxSize - Variable in class org.jwat.warc.WarcReader

Max size allowed for a WARC header.

warcIdentifiedPayloadType - Variable in class org.jwat.warc.WarcHeader

warcIdentifiedPayloadTypeStr - Variable in class org.jwat.warc.WarcHeader

warcInetAddress - Variable in class org.jwat.warc.WarcHeader

warcIpAddress - Variable in class org.jwat.warc.WarcHeader

warcPayloadDigest - Variable in class org.jwat.warc.WarcHeader

warcPayloadDigestStr - Variable in class org.jwat.warc.WarcHeader

warcProfileIdx - Variable in class org.jwat.warc.WarcHeader

warcProfileStr - Variable in class org.jwat.warc.WarcHeader

WarcReader - Class in org.jwat.warc

Base class for WARC reader implementations.

WarcReader() - Constructor for class org.jwat.warc.WarcReader

WarcReaderCompressed - Class in org.jwat.warc

WARC Reader implementation for reading GZip compressed files.

WarcReaderFactory - Class in org.jwat.warc

Factory used for creating WarcReader instances.

WarcReaderFactory() - Constructor for class org.jwat.warc.WarcReaderFactory

Private constructor to enforce factory method.

WarcReaderUncompressed - Class in org.jwat.warc

WARC Reader implementation for reading uncompressed files.

warcRecord - Variable in class org.jwat.warc.WarcReader

Current WARC record object.

WarcRecord - Class in org.jwat.warc

This class represents a parsed WARC record header block including possible validation and format warnings/errors encountered in the process.

WarcRecord() - Constructor for class org.jwat.warc.WarcRecord

Non public constructor to allow unit testing.

warcRecordIdStr - Variable in class org.jwat.warc.WarcHeader

warcRecordIdUri - Variable in class org.jwat.warc.WarcHeader

warcRefersToStr - Variable in class org.jwat.warc.WarcHeader

warcRefersToUri - Variable in class org.jwat.warc.WarcHeader

warcSegmentNumber - Variable in class org.jwat.warc.WarcHeader

warcSegmentNumberStr - Variable in class org.jwat.warc.WarcHeader

warcSegmentOriginIdStr - Variable in class org.jwat.warc.WarcHeader

warcSegmentOriginIdUrl - Variable in class org.jwat.warc.WarcHeader

warcSegmentTotalLength - Variable in class org.jwat.warc.WarcHeader

warcSegmentTotalLengthStr - Variable in class org.jwat.warc.WarcHeader

warcTargetUriStr - Variable in class org.jwat.warc.WarcHeader

warcTargetUriUri - Variable in class org.jwat.warc.WarcHeader

warcTruncatedIdx - Variable in class org.jwat.warc.WarcHeader

warcTruncatedStr - Variable in class org.jwat.warc.WarcHeader

warcTypeIdx - Variable in class org.jwat.warc.WarcHeader

warcTypeStr - Variable in class org.jwat.warc.WarcHeader

warcWarcinfoIdStr - Variable in class org.jwat.warc.WarcHeader

warcWarcInfoIdUri - Variable in class org.jwat.warc.WarcHeader

WarcWriter - Class in org.jwat.warc

Base class for WARC writer implementations.

WarcWriter() - Constructor for class org.jwat.warc.WarcWriter

WarcWriterCompressed - Class in org.jwat.warc

WARC Writer implementation for writing GZip compressed files.

WarcWriterFactory - Class in org.jwat.warc

Factory used for creating WarcWriter instances.

WarcWriterFactory() - Constructor for class org.jwat.warc.WarcWriterFactory

Private constructor to enforce factory method.

WarcWriterUncompressed - Class in org.jwat.warc

WARC Writer implementation for writing uncompressed files.

warnings - Variable in class org.jwat.warc.WarcReader

Aggregate number of warnings encountered while parsing.

writeHeader(byte[], Long) - Method in class org.jwat.warc.WarcWriter

Write a raw WARC header to the WARC output stream.

writeHeader(WarcRecord) - Method in class org.jwat.warc.WarcWriter

Write a WARC header to the WARC output stream.

writeHeader(byte[], Long) - Method in class org.jwat.warc.WarcWriterCompressed

writeHeader(WarcRecord) - Method in class org.jwat.warc.WarcWriterCompressed

writeHeader(WarcRecord) - Method in class org.jwat.warc.WarcWriterUncompressed

writeHeader_impl(WarcRecord) - Method in class org.jwat.warc.WarcWriter

Write a WARC header to the WARC output stream.

writePayload(byte[]) - Method in class org.jwat.warc.WarcWriter

writePayload(byte[], int, int) - Method in class org.jwat.warc.WarcWriter

writePayload(byte[]) - Method in class org.jwat.warc.WarcWriterCompressed

writePayload(byte[], int, int) - Method in class org.jwat.warc.WarcWriterCompressed

writer - Variable in class org.jwat.warc.WarcWriterCompressed

GZip Writer used.

A B C D E F G H I L M O P R S T W