me.vaperion.blade.tokenizer

Class AbstractStringTokenizer

java.lang.Object

me.vaperion.blade.tokenizer.AbstractStringTokenizer

Direct Known Subclasses:

StringTokenizer

public abstract class AbstractStringTokenizer
extends java.lang.Object

A simple class to help tokenize strings.

Field Summary

Fields

Modifier and Type	Field and Description
static char	BACKTICK
static char	DOUBLE_QUOTE
static java.util.function.Predicate<java.lang.Character>	DOUBLE_QUOTE_PRED
static char	ESCAPE
static java.util.function.Predicate<java.lang.Character>	ESCAPE_PRED
static java.util.function.Predicate<java.lang.Character>	QUOTE_PRED
static char	SINGLE_QUOTE
static java.util.function.Predicate<java.lang.Character>	SINGLE_QUOTE_PRED
static java.util.function.Predicate<java.lang.Character>	WHITESPACE_PRED

Constructor Summary

Constructors

Constructor and Description
AbstractStringTokenizer(java.lang.String string)

Method Summary

Modifier and Type	Method and Description
@NotNull java.lang.String	consumed() Returns the consumed characters as a string.
int	cursorPosition() Returns the current cursor position.
void	dropSavedCursor() Drops the saved cursor position.
void	expect(char expected) Expects the current character to be the given character.
void	expect(@NotNull java.util.function.Predicate<java.lang.Character> predicate) Expects the current character to pass the given test.
void	expectWhitespace() Expects the current character to be whitespace.
boolean	hasNext() Returns whether the tokenizer has remaining characters.
boolean	hasNext(int offset) Returns whether the tokenizer has remaining characters at the given offset.
int	length() Returns the original length of the string.
char	peek() Returns the character at the current cursor position without consuming it.
boolean	peek(char c) Checks if the current character matches the given character.
char	peek(int offset) Returns the character at the given offset from the current cursor position without consuming it.
boolean	peek(@NotNull java.util.function.Predicate<java.lang.Character> predicate) Checks if the current character matches the given predicate.
boolean	peekWhitespace() Checks if the current character is a whitespace character.
@NotNull java.lang.String	remaining() Returns the remaining characters as a string.
boolean	remainingContainsWhitespace() Checks if the remaining string contains whitespace anywhere.
int	remainingLength() Returns the number of remaining characters.
void	require(int count) Ensures that at least the specified number of characters remain.
void	resetCursor() Resets the cursor position to the beginning.
void	restoreCursor() Restores the cursor to the last saved position.
void	saveCursor() Saves the current cursor position.
void	setCursor(int index) Sets the cursor position to the specified index.
void	setString(@NotNull java.lang.String string) Sets the string to tokenize.
void	skip() Skips one character.
void	skip(int offset) Skips the specified number of characters.
void	skipUntil(char c) Skips all characters until the given character is found.
void	skipUntil(java.util.function.Predicate<java.lang.Character> predicate) Skips all characters until the given predicate is matched.
void	skipWhile(char c) Skips all consecutive occurrences of the given character.
void	skipWhile(java.util.function.Predicate<java.lang.Character> c) Skips all consecutive characters matching the given predicate.
void	skipWhitespace() Skips all consecutive whitespace characters.
@NotNull java.lang.String	string() Returns the string to tokenize.
char	take() Consumes a single character at the current cursor position.
@NotNull java.lang.String	take(int offset) Consumes a specified number of characters.
@Nullable java.lang.String	takeQuotedString() Consumes a quoted (single or double) or unquoted string.
@Nullable java.lang.String	takeQuotedString(boolean lenient) Consumes a quoted (single or double) or unquoted string.
@Nullable java.lang.String	takeString(@NotNull java.util.function.Predicate<java.lang.Character> predicate) Consumes a quoted or unquoted string based on the given quote predicate.
@Nullable java.lang.String	takeString(@NotNull java.util.function.Predicate<java.lang.Character> predicate, boolean allowEmpty, boolean allowUnterminated) Consumes a quoted or unquoted string based on the given quote predicate.
@NotNull java.lang.String	takeUnquotedString() Consumes an unquoted string (until whitespace or end of input).
@Nullable java.lang.String	takeUntil(boolean allowUnterminated, char terminator) Consumes all characters until the terminator character is found.
@Nullable java.lang.String	takeUntil(char terminator) Consumes all characters until the terminator character is found.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ESCAPE

public static final char ESCAPE

See Also:

Constant Field Values

BACKTICK

public static final char BACKTICK

See Also:

Constant Field Values

SINGLE_QUOTE

public static final char SINGLE_QUOTE

See Also:

Constant Field Values

DOUBLE_QUOTE

public static final char DOUBLE_QUOTE

See Also:

Constant Field Values

ESCAPE_PRED

public static final java.util.function.Predicate<java.lang.Character> ESCAPE_PRED

WHITESPACE_PRED

public static final java.util.function.Predicate<java.lang.Character> WHITESPACE_PRED

SINGLE_QUOTE_PRED

public static final java.util.function.Predicate<java.lang.Character> SINGLE_QUOTE_PRED

DOUBLE_QUOTE_PRED

public static final java.util.function.Predicate<java.lang.Character> DOUBLE_QUOTE_PRED

QUOTE_PRED

public static final java.util.function.Predicate<java.lang.Character> QUOTE_PRED

Constructor Detail

AbstractStringTokenizer

public AbstractStringTokenizer(java.lang.String string)

Method Detail

string

@NotNull
public @NotNull java.lang.String string()

Returns the string to tokenize.

Returns:

the string to tokenize

setString

public void setString(@NotNull
                      @NotNull java.lang.String string)

Sets the string to tokenize.

Parameters:

string - the string to tokenize

take

public char take()
          throws TokenizerError

Consumes a single character at the current cursor position.

Returns:

the character consumed

Throws:

TokenizerError

expectWhitespace

public void expectWhitespace()
                      throws TokenizerError

Expects the current character to be whitespace.

Throws:

TokenizerError

expect

public void expect(char expected)
            throws TokenizerError

Expects the current character to be the given character.

Parameters:

expected - the expected character

Throws:

TokenizerError

expect

public void expect(@NotNull
                   @NotNull java.util.function.Predicate<java.lang.Character> predicate)
            throws TokenizerError

Expects the current character to pass the given test.

Parameters:

predicate - the predicate to test the character

Throws:

TokenizerError

hasNext

public boolean hasNext()

Returns whether the tokenizer has remaining characters.

Returns:

true if there are remaining characters

hasNext

public boolean hasNext(int offset)

Returns whether the tokenizer has remaining characters at the given offset.

Parameters:

offset - the offset from the current cursor position

Returns:

true if there are remaining characters at the offset

require

public void require(int count)
             throws TokenizerError

Ensures that at least the specified number of characters remain.

Parameters:

count - the minimum number of characters required

Throws:

TokenizerError

peek

public char peek()
          throws TokenizerError

Returns the character at the current cursor position without consuming it.

Returns:

the character at the current position

Throws:

TokenizerError

peek

public char peek(int offset)
          throws TokenizerError

Returns the character at the given offset from the current cursor position without consuming it.

Parameters:

offset - the offset from the current cursor position (can be negative)

Returns:

the character at the offset

Throws:

TokenizerError

peekWhitespace

public boolean peekWhitespace()
                       throws TokenizerError

Checks if the current character is a whitespace character.

Throws:

TokenizerError

peek

public boolean peek(char c)
             throws TokenizerError

Checks if the current character matches the given character.

Parameters:

c - the character to match

Throws:

TokenizerError

peek

public boolean peek(@NotNull
                    @NotNull java.util.function.Predicate<java.lang.Character> predicate)
             throws TokenizerError

Checks if the current character matches the given predicate.

Parameters:

predicate - the predicate to test the character

Throws:

TokenizerError

remainingContainsWhitespace

public boolean remainingContainsWhitespace()

Checks if the remaining string contains whitespace anywhere.

Returns:

true if the remaining string contains whitespace

skip

public void skip()
          throws TokenizerError

Skips one character.

Throws:

TokenizerError

skip

public void skip(int offset)
          throws TokenizerError

Skips the specified number of characters.

Parameters:

offset - the amount to skip (can be negative)

Throws:

TokenizerError

take

@NotNull
public @NotNull java.lang.String take(int offset)
                                        throws TokenizerError

Consumes a specified number of characters.

Parameters:

offset - the number of characters to consume

Returns:

the consumed string

Throws:

TokenizerError

resetCursor

public void resetCursor()

Resets the cursor position to the beginning.

setCursor

public void setCursor(int index)

Sets the cursor position to the specified index.

Parameters:

index - the new cursor position

saveCursor

public void saveCursor()

Saves the current cursor position.

restoreCursor

public void restoreCursor()

Restores the cursor to the last saved position.

dropSavedCursor

public void dropSavedCursor()

Drops the saved cursor position.

consumed

@NotNull
public @NotNull java.lang.String consumed()

Returns the consumed characters as a string.

Returns:

the consumed characters as a string

remaining

@NotNull
public @NotNull java.lang.String remaining()

Returns the remaining characters as a string.

Returns:

the remaining characters as a string

cursorPosition

public int cursorPosition()

Returns the current cursor position.

Returns:

the cursor position

remainingLength

public int remainingLength()

Returns the number of remaining characters.

Returns:

the number of remaining characters

length

public int length()

Returns the original length of the string.

Returns:

the original length of the string

skipWhile

public void skipWhile(char c)

Skips all consecutive occurrences of the given character.

Parameters:

c - the character to skip

skipWhile

public void skipWhile(java.util.function.Predicate<java.lang.Character> c)

Skips all consecutive characters matching the given predicate.

Parameters:

c - the predicate to test the characters

skipUntil

public void skipUntil(char c)

Skips all characters until the given character is found.

Parameters:

c - the character to stop at

skipUntil

public void skipUntil(java.util.function.Predicate<java.lang.Character> predicate)

Skips all characters until the given predicate is matched.

Parameters:

predicate - the predicate to test the characters

skipWhitespace

public void skipWhitespace()

Skips all consecutive whitespace characters.

takeQuotedString

@Nullable
public @Nullable java.lang.String takeQuotedString()

Consumes a quoted (single or double) or unquoted string.

Returns:

the consumed string, or null if there's nothing left or if the quoted string is not terminated

takeQuotedString

@Nullable
public @Nullable java.lang.String takeQuotedString(boolean lenient)

Consumes a quoted (single or double) or unquoted string.

Parameters:

lenient - whether to allow unterminated quoted strings

Returns:

the consumed string, or null if there's nothing left or if the quoted string is not terminated (and `lenient` is false)

takeString

@Nullable
public @Nullable java.lang.String takeString(@NotNull
                                                       @NotNull java.util.function.Predicate<java.lang.Character> predicate)

Consumes a quoted or unquoted string based on the given quote predicate.

Parameters:

predicate - the predicate to test for quote characters

Returns:

the consumed string, or null if there's nothing left or if the quoted string is not terminated

takeString

@Nullable
public @Nullable java.lang.String takeString(@NotNull
                                                       @NotNull java.util.function.Predicate<java.lang.Character> predicate,
                                                       boolean allowEmpty,
                                                       boolean allowUnterminated)

Consumes a quoted or unquoted string based on the given quote predicate.

Parameters:

predicate - the predicate to test for quote characters

allowEmpty - whether to allow empty strings

allowUnterminated - whether to allow unterminated quoted strings

Returns:

the consumed string, or null if there's nothing left (and `allowEmpty` is false) or if the quoted string is not terminated (and `allowUnterminated` is false)

takeUnquotedString

@NotNull
public @NotNull java.lang.String takeUnquotedString()

Consumes an unquoted string (until whitespace or end of input).

Returns:

the consumed unquoted string

takeUntil

@Nullable
public @Nullable java.lang.String takeUntil(char terminator)

Consumes all characters until the terminator character is found. Supports escaping the terminator with a backslash.

Parameters:

terminator - the terminator character

Returns:

the consumed string, or null if the sequence is not properly terminated

takeUntil

@Nullable
public @Nullable java.lang.String takeUntil(boolean allowUnterminated,
                                                      char terminator)

Consumes all characters until the terminator character is found. Supports escaping the terminator with a backslash.

Parameters:

allowUnterminated - whether to allow unterminated sequences

terminator - the terminator character

Returns:

the consumed string, or null if the sequence is not properly terminated (and `allowUnterminated` is false)