Package org.spf4j.log

Class AvroDataFileAppender

java.lang.Object

ch.qos.logback.core.spi.ContextAwareBase

ch.qos.logback.core.UnsynchronizedAppenderBase<ch.qos.logback.classic.spi.ILoggingEvent>

org.spf4j.log.AvroDataFileAppender

All Implemented Interfaces:

ch.qos.logback.core.Appender<ch.qos.logback.classic.spi.ILoggingEvent>, ch.qos.logback.core.spi.ContextAware, ch.qos.logback.core.spi.FilterAttachable<ch.qos.logback.classic.spi.ILoggingEvent>, ch.qos.logback.core.spi.LifeCycle

public final class AvroDataFileAppender
extends ch.qos.logback.core.UnsynchronizedAppenderBase<ch.qos.logback.classic.spi.ILoggingEvent>

an Appender that will log into binary avro data files. Files will be be partition by day. Day boundaries are determined by the provided zone id.

Author:

Zoltan Farkas

Field Summary

Fields inherited from class ch.qos.logback.core.UnsynchronizedAppenderBase

name, started

Fields inherited from class ch.qos.logback.core.spi.ContextAwareBase

context

Constructor Summary

Constructors

Constructor	Description
AvroDataFileAppender()

Method Summary

Modifier and Type	Method	Description
protected void	append(ch.qos.logback.classic.spi.ILoggingEvent eventObject)
void	cleanup()
long	flush()
Path	getCurrentFile()
long	getCurrentFileNrLogs()
org.apache.avro.file.FileReader<org.spf4j.base.avro.LogRecord>	getCurrentLogs()
Path	getDestinationPath()
void	getFilteredLogs(String originPrefix, long ptailOffset, long limit, Predicate<org.spf4j.base.avro.LogRecord> pred, Consumer<org.spf4j.base.avro.LogRecord> records)
List<Path>	getLogFiles()
void	getLogs(String originPrefix, long ptailOffset, long limit, Consumer<org.spf4j.base.avro.LogRecord> records)
long	getNrLogs()
static long	getNrLogs(Path file)
List<Path>	getOldLogFiles()
static boolean	isValidFile(Path file)
static void	readLogs(org.spf4j.log.AvroDataFileAppender.LogFileRange fileRange, org.apache.avro.io.DatumReader<org.spf4j.base.avro.LogRecord> reader, String originPrefix, Consumer<org.spf4j.base.avro.LogRecord> records)
void	setAppendRetries(int appendRetries)	When serializing a object a ConcurrentModificationEnception can be thrown if this is mutated by another thread.
void	setCodec(String codec)
void	setDestinationPath(String destinationPath)
void	setFileNameBase(String fileNameBase)
void	setMaxLogsBytes(long maxLogsBytes)
void	setMaxNrFiles(int maxNrFiles)
void	setPartitionZoneID(String zoneIdStr)
static void	skip(org.apache.avro.file.DataFileStream<org.spf4j.base.avro.LogRecord> it, long count)
void	start()
void	stop()
String	toString()

Methods inherited from class ch.qos.logback.core.UnsynchronizedAppenderBase

addFilter, clearAllFilters, doAppend, getCopyOfAttachedFiltersList, getFilterChainDecision, getName, isStarted, setName

Methods inherited from class ch.qos.logback.core.spi.ContextAwareBase

addError, addError, addInfo, addInfo, addStatus, addWarn, addWarn, getContext, getDeclaredOrigin, getStatusManager, setContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface ch.qos.logback.core.spi.ContextAware

addError, addError, addInfo, addInfo, addStatus, addWarn, addWarn, getContext, setContext

Constructor Detail

AvroDataFileAppender

public AvroDataFileAppender()

Method Detail

setAppendRetries

public void setAppendRetries(int appendRetries)

When serializing a object a ConcurrentModificationEnception can be thrown if this is mutated by another thread. (async logging) this can re-tried, a few times, and hopefully log this successfully.

Parameters:

appendRetries -

setMaxNrFiles

public void setMaxNrFiles(int maxNrFiles)

setMaxLogsBytes

public void setMaxLogsBytes(long maxLogsBytes)

setCodec

public void setCodec(String codec)

setFileNameBase

public void setFileNameBase(String fileNameBase)

setDestinationPath

public void setDestinationPath(String destinationPath)

setPartitionZoneID

public void setPartitionZoneID(String zoneIdStr)

getDestinationPath

public Path getDestinationPath()

cleanup

public void cleanup()
             throws IOException

Throws:

IOException

getLogFiles

public List<Path> getLogFiles()
                       throws IOException

Returns:

all log files in chronological order.

Throws:

IOException

getOldLogFiles

public List<Path> getOldLogFiles()
                          throws IOException

Throws:

IOException

getCurrentFile

public Path getCurrentFile()

flush

public long flush()
           throws IOException

Throws:

IOException

getCurrentFileNrLogs

public long getCurrentFileNrLogs()
                          throws IOException

Throws:

IOException

getNrLogs

public long getNrLogs()
               throws IOException

Throws:

IOException

getCurrentLogs

public org.apache.avro.file.FileReader<org.spf4j.base.avro.LogRecord> getCurrentLogs()
                                                                              throws IOException

Throws:

IOException

getLogs

public void getLogs(String originPrefix,
                    long ptailOffset,
                    long limit,
                    Consumer<org.spf4j.base.avro.LogRecord> records)
             throws IOException

Throws:

IOException

readLogs

public static void readLogs(org.spf4j.log.AvroDataFileAppender.LogFileRange fileRange,
                            org.apache.avro.io.DatumReader<org.spf4j.base.avro.LogRecord> reader,
                            String originPrefix,
                            Consumer<org.spf4j.base.avro.LogRecord> records)
                     throws IOException

Throws:

IOException

getFilteredLogs

public void getFilteredLogs(String originPrefix,
                            long ptailOffset,
                            long limit,
                            Predicate<org.spf4j.base.avro.LogRecord> pred,
                            Consumer<org.spf4j.base.avro.LogRecord> records)
                     throws IOException

Throws:

IOException

skip

public static void skip(org.apache.avro.file.DataFileStream<org.spf4j.base.avro.LogRecord> it,
                        long count)
                 throws IOException

Throws:

IOException

getNrLogs

public static long getNrLogs(Path file)
                      throws IOException

Throws:

IOException

stop

public void stop()

Specified by:

stop in interface ch.qos.logback.core.spi.LifeCycle

Overrides:

stop in class ch.qos.logback.core.UnsynchronizedAppenderBase<ch.qos.logback.classic.spi.ILoggingEvent>

start

public void start()

Specified by:

start in interface ch.qos.logback.core.spi.LifeCycle

Overrides:

start in class ch.qos.logback.core.UnsynchronizedAppenderBase<ch.qos.logback.classic.spi.ILoggingEvent>

isValidFile

public static boolean isValidFile(Path file)
                           throws IOException

Throws:

IOException

append

protected void append(ch.qos.logback.classic.spi.ILoggingEvent eventObject)

Specified by:

append in class ch.qos.logback.core.UnsynchronizedAppenderBase<ch.qos.logback.classic.spi.ILoggingEvent>

toString

public String toString()

Overrides:

toString in class ch.qos.logback.core.UnsynchronizedAppenderBase<ch.qos.logback.classic.spi.ILoggingEvent>