Package org.graylog2.plugin

Class Message

java.lang.Object
org.graylog2.plugin.Message
All Implemented Interfaces:
Iterable<Message>, Indexable, Messages, Acknowledgeable
@NotThreadSafe public class Message extends Object implements Messages, Indexable, Acknowledgeable

  • Field Details

    • FIELD_ID

      public static final String FIELD_ID
      The "_id" is used as document ID to address the document in Elasticsearch. TODO: We might want to use the "gl2_message_id" for this in the future to reduce storage and avoid having basically two different message IDs. To do that we have to check if switching to a different ID format breaks anything with regard to expectations in other code and existing data in Elasticsearch.
      See Also:

    • FIELD_INDEX

      public static final String FIELD_INDEX
      See Also:

    • FIELD_MESSAGE

      public static final String FIELD_MESSAGE
      See Also:

    • FIELD_FULL_MESSAGE

      public static final String FIELD_FULL_MESSAGE
      See Also:

    • FIELD_SOURCE

      public static final String FIELD_SOURCE
      See Also:

    • FIELD_TIMESTAMP

      public static final String FIELD_TIMESTAMP
      See Also:

    • FIELD_LEVEL

      public static final String FIELD_LEVEL
      See Also:

    • FIELD_STREAMS

      public static final String FIELD_STREAMS
      See Also:

    • FIELD_FAILED_MESSAGE_STREAMS

      public static final String FIELD_FAILED_MESSAGE_STREAMS
      See Also:

    • INTERNAL_FIELD_PREFIX

      public static final String INTERNAL_FIELD_PREFIX
      Graylog is writing internal metadata to messages using this field prefix. Users must not use this prefix for custom message fields.
      See Also:

    • FIELD_GL2_ACCOUNTED_MESSAGE_SIZE

      public static final String FIELD_GL2_ACCOUNTED_MESSAGE_SIZE
      Will be set to the accounted message size in bytes.
      See Also:

    • FIELD_GL2_MESSAGE_ID

      public static final String FIELD_GL2_MESSAGE_ID
      This is the message ID. It will be set to a ULID during processing.

      Attention: This is currently NOT the "_id" field which is used as ID for the document in Elasticsearch!

      Implementation notes

      We are not using the UUID in "_id" for this field because of the following reasons:
      • Using ULIDs results in shorter IDs (26 characters for ULID vs 36 for UUID) and thus reduced storage usage
      • They are guaranteed to be lexicographically sortable (UUIDs are only lexicographically sortable when time-based ones are used)
      See: ...
      See Also:

    • GL2_SECOND_SORT_FIELD

      public static final String GL2_SECOND_SORT_FIELD
      Not a field, but an alias to the field we automatically use to determine the second sort order. This currently points to gl2_message_id.
      See Also:

    • FIELD_GL2_PROCESSING_ERROR

      public static final String FIELD_GL2_PROCESSING_ERROR
      Can be set to indicate a message processing error. (e.g. set by the pipeline interpreter when an error occurs)
      See Also:

    • FIELD_GL2_PROCESSING_TIMESTAMP

      public static final String FIELD_GL2_PROCESSING_TIMESTAMP
      Will be set to the message processing time after all message processors have been run.
      See Also:

    • FIELD_GL2_RECEIVE_TIMESTAMP

      public static final String FIELD_GL2_RECEIVE_TIMESTAMP
      Will be set to the message receive time at the input.
      See Also:

    • FIELD_GL2_PROCESSING_DURATION_MS

      public static final String FIELD_GL2_PROCESSING_DURATION_MS
      Reflects the time span from receiving the message till sending it to the output in milliseconds. Will be set after all message processors have been run.
      See Also:

    • FIELD_GL2_REMOTE_HOSTNAME

      public static final String FIELD_GL2_REMOTE_HOSTNAME
      Will be set to the hostname of the source node that sent a message. (if reverse lookup is enabled)
      See Also:

    • FIELD_GL2_REMOTE_IP

      public static final String FIELD_GL2_REMOTE_IP
      Will be set to the IP address of the source node that sent a message.
      See Also:

    • FIELD_GL2_REMOTE_PORT

      public static final String FIELD_GL2_REMOTE_PORT
      Will be set to the socket port of the source node that sent a message.
      See Also:

    • FIELD_GL2_SOURCE_COLLECTOR

      public static final String FIELD_GL2_SOURCE_COLLECTOR
      Can be set to the collector ID that sent a message. (e.g. used in the beats codec)
      See Also:

    • FIELD_GL2_SOURCE_COLLECTOR_INPUT

      @Deprecated public static final String FIELD_GL2_SOURCE_COLLECTOR_INPUT
      Deprecated.
      This was used in the legacy collector/sidecar system and contained the database ID of the collector input.
      See Also:

    • FIELD_GL2_SOURCE_INPUT

      public static final String FIELD_GL2_SOURCE_INPUT
      Will be set to the ID of the input that received the message.
      See Also:

    • FIELD_GL2_FORWARDER_INPUT

      public static final String FIELD_GL2_FORWARDER_INPUT
      See Also:

    • FIELD_GL2_SOURCE_NODE

      public static final String FIELD_GL2_SOURCE_NODE
      Will be set to the ID of the node that received the message.
      See Also:

    • FIELD_GL2_SOURCE_RADIO

      @Deprecated public static final String FIELD_GL2_SOURCE_RADIO
      Deprecated.
      This was used with the now removed radio system and contained the ID of a radio node. TODO: Due to be removed in Graylog 3.x
      See Also:

    • FIELD_GL2_SOURCE_RADIO_INPUT

      @Deprecated public static final String FIELD_GL2_SOURCE_RADIO_INPUT
      Deprecated.
      This was used with the now removed radio system and contained the input ID of a radio node. TODO: Due to be removed in Graylog 3.x
      See Also:

    • SEARCHABLE_ES_FIELDS

      public static final Set<String> SEARCHABLE_ES_FIELDS

    • RESERVED_SETTABLE_FIELDS

      public static final com.google.common.collect.ImmutableSet<String> RESERVED_SETTABLE_FIELDS

    • FIELDS_UNCHANGEABLE_BY_CUSTOM_MAPPINGS

      public static final com.google.common.collect.ImmutableSet<String> FIELDS_UNCHANGEABLE_BY_CUSTOM_MAPPINGS

    • RESERVED_FIELDS

      public static final com.google.common.collect.ImmutableSet<String> RESERVED_FIELDS

    • FILTERED_FIELDS

      public static final com.google.common.collect.ImmutableSet<String> FILTERED_FIELDS

    • ID_FUNCTION

      @Deprecated public static final com.google.common.base.Function<Message,String> ID_FUNCTION
      Deprecated.

  • Method Details

    • isComplete

      public boolean isComplete()

    • getId

      public String getId()
      Description copied from interface: Indexable
      Returns the id to address the document in Elasticsearch. Depending on the implementation this might return a UUID or ULID This method should only be used where backwards compatibility is needed. Newer code should use Indexable.getMessageId() instead.
      Specified by:
      getId in interface Indexable

    • getMessageId

      public String getMessageId()
      Description copied from interface: Indexable
      Returns the id to address the document in Elasticsearch. The message id is represented as a ULID
      Specified by:
      getMessageId in interface Indexable

    • getTimestamp

      public org.joda.time.DateTime getTimestamp()
      Specified by:
      getTimestamp in interface Indexable

    • toElasticSearchObject

      public Map<String,Object> toElasticSearchObject(com.fasterxml.jackson.databind.ObjectMapper objectMapper, @Nonnull com.codahale.metrics.Meter invalidTimestampMeter)
      Specified by:
      toElasticSearchObject in interface Indexable

    • ensureValidTimestamp

      public void ensureValidTimestamp()

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • toDumpString

      public String toDumpString()

    • getMessage

      public String getMessage()

    • getSource

      public String getSource()

    • setSource

      public void setSource(String source)

    • addField

      public void addField(String key, Object value)

    • getSize

      public long getSize()
      Specified by:
      getSize in interface Indexable

    • validKey

      public static boolean validKey(String key)

    • cleanKey

      public static String cleanKey(String key)

    • addFields

      public void addFields(Map<String,Object> fields)

    • removeField

      public void removeField(String key)

    • getFieldAs

      public <T> T getFieldAs(Class<T> T, String key) throws ClassCastException
      Throws:
      ClassCastException

    • getField

      public Object getField(String key)

    • getFields

      public Map<String,Object> getFields()

    • getFieldsEntries

      public Iterable<Map.Entry<String,Object>> getFieldsEntries()

    • getFieldCount

      public int getFieldCount()

    • hasField

      public boolean hasField(String field)

    • getFieldNames

      public Set<String> getFieldNames()

    • setStreams

      @Deprecated public void setStreams(List<Stream> streams)
      Deprecated.

    • getStreams

      public Set<Stream> getStreams()
      Get the streams this message is currently routed to.
      Returns:
      an immutable copy of the current set of assigned streams, empty if no streams have been assigned

    • addStream

      public void addStream(Stream stream)
      Assign the given stream to this message.
      Parameters:
      stream - the stream to route this message into

    • addStreams

      public void addStreams(Iterable<Stream> newStreams)
      Assign all of the streams to this message.
      Parameters:
      newStreams - an iterable of Stream objects

    • removeStream

      public boolean removeStream(Stream stream)
      Remove the stream assignment from this message.
      Parameters:
      stream - the stream assignment to remove this message from
      Returns:
      true if this message was assigned to the stream

    • getIndexSets

      public Set<IndexSet> getIndexSets()
      Return the index sets for this message based on the assigned streams.
      Returns:
      index sets

    • getStreamIds

      public Set<String> getStreamIds()

    • setFilterOut

      public void setFilterOut(boolean filterOut)

    • getFilterOut

      public boolean getFilterOut()

    • setSourceInputId

      public void setSourceInputId(String sourceInputId)

    • getSourceInputId

      public String getSourceInputId()

    • getIsSourceInetAddress

      @Deprecated public boolean getIsSourceInetAddress()
      Deprecated.

    • getInetAddress

      public InetAddress getInetAddress()

    • setJournalOffset

      public void setJournalOffset(long journalOffset)

    • getJournalOffset

      @Deprecated public long getJournalOffset()
      Deprecated.
      Use getMessageQueueId() instead.

    • setMessageQueueId

      public void setMessageQueueId(Object messageQueueId)

    • setSequenceNr

      public void setSequenceNr(int sequenceNr)

    • getSequenceNr

      public int getSequenceNr()

    • getMessageQueueId

      @Nullable public Object getMessageQueueId()
      Specified by:
      getMessageQueueId in interface Acknowledgeable

    • getReceiveTime

      @Nullable public org.joda.time.DateTime getReceiveTime()
      Specified by:
      getReceiveTime in interface Indexable

    • setReceiveTime

      public void setReceiveTime(org.joda.time.DateTime receiveTime)
      Sets the Message receive time. The time is taken from the receive time of the message, before it was written to the journal.
      Parameters:
      receiveTime - the new receive timestamp

    • getProcessingTime

      @Nullable public org.joda.time.DateTime getProcessingTime()

    • setProcessingTime

      public void setProcessingTime(org.joda.time.DateTime processingTime)
      Sets the Message processing Time. The processing time should only be set once all message processors have finished. This is usually done in the ProcessBufferProcessor
      Parameters:
      processingTime - the new processing timestamp

    • recordTiming

      public void recordTiming(ServerStatus serverStatus, String name, long elapsedNanos)

    • recordCounter

      public void recordCounter(ServerStatus serverStatus, String name, int counter)

    • recordingsAsString

      public String recordingsAsString()

    • hasRecordings

      public boolean hasRecordings()

    • addProcessingError

      public void addProcessingError(@Nonnull Message.ProcessingError processingError)
      Appends another processing error.
      Parameters:
      processingError - another processing error to be appended. Must not be null.

    • processingErrors

      public List<Message.ProcessingError> processingErrors()
      Returns a list of submitted processing errors

    • iterator

      @Nonnull public Iterator<Message> iterator()
      Specified by:
      iterator in interface Iterable<Message>

    • supportsFailureHandling

      public boolean supportsFailureHandling()
      Description copied from interface: Indexable
      Guides the failure handling framework when deciding whether this particular message should be accepted for the further failure processing. By default disabled for all messages.
      Specified by:
      supportsFailureHandling in interface Indexable

    • setMetadata

      public void setMetadata(String key, Object value)
      Store the specified metadata value within the message's internal metadata map.
      Parameters:
      key - A globally unique string key identifier for the metadata value.
      value - The metadata object value.

    • getMetadataValue

      @Nullable public Object getMetadataValue(String key)
      Get the metadata value for the specified key.
      Parameters:
      key - The string key for the metadata entry.

    • getMetadataValue

      @Nullable public Object getMetadataValue(String key, Object defaultValue)
      Get the metadata value for the specified key. If not present, then return the default value.
      Parameters:
      key - The string key for the metadata entry.

    • removeMetadata

      public void removeMetadata(String key)
      Remove the metadata value for the specified key.
      Parameters:
      key - The string key for the metadata entry.