wyc.io

Class WhileyFileLexer

java.lang.Object

wyc.io.WhileyFileLexer

public class WhileyFileLexer
extends Object

Split a source file into a list of tokens. These tokens can then be fed into the parser in order to generate an Abstract Syntax Tree (AST).

Author:

David J. Pearce

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	WhileyFileLexer.Token The base class for all tokens.

Field Summary

Fields

Modifier and Type	Field and Description
static HashMap<String,WhileyFileLexer.Token.Kind>	keywords A map from identifier strings to the corresponding token kind.
static char	UC_ELEMENTOF
static char	UC_EMPTYSET
static char	UC_EXISTS
static char	UC_FORALL
static char	UC_GREATEREQUALS
static char	UC_LESSEQUALS
static char	UC_LOGICALAND
static char	UC_LOGICALOR
static char	UC_SETINTERSECTION
static char	UC_SETUNION
static char	UC_SUBSET
static char	UC_SUBSETEQ
static char	UC_SUPSET
static char	UC_SUPSETEQ

Constructor Summary

Constructors

Constructor and Description
WhileyFileLexer(wyfs.lang.Path.Entry<WhileyFile> entry)

Method Summary

Modifier and Type	Method and Description
boolean	isOperatorStart(char c)
List<WhileyFileLexer.Token>	scan() Scan all characters from the input stream and generate a corresponding list of tokens, whilst discarding all whitespace and comments.
WhileyFileLexer.Token	scanBlockComment()
WhileyFileLexer.Token	scanCharacterConstant() Scan a character constant, such as e.g.
WhileyFileLexer.Token	scanIdentifier()
WhileyFileLexer.Token	scanIndent() Scan one or more spaces or tab characters, combining them to form an "indent".
WhileyFileLexer.Token	scanLineComment()
WhileyFileLexer.Token	scanNumericConstant() Scan a numeric constant.
WhileyFileLexer.Token	scanOperator()
WhileyFileLexer.Token	scanStringConstant()
void	scanWhiteSpace(List<WhileyFileLexer.Token> tokens)
void	skipWhitespace(List<WhileyFileLexer.Token> tokens) Skip over any whitespace at the current index position in the input string.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

UC_FORALL

public static final char UC_FORALL

See Also:

Constant Field Values

UC_EXISTS

public static final char UC_EXISTS

See Also:

Constant Field Values

UC_EMPTYSET

public static final char UC_EMPTYSET

See Also:

Constant Field Values

UC_SUBSET

public static final char UC_SUBSET

See Also:

Constant Field Values

UC_SUBSETEQ

public static final char UC_SUBSETEQ

See Also:

Constant Field Values

UC_SUPSET

public static final char UC_SUPSET

See Also:

Constant Field Values

UC_SUPSETEQ

public static final char UC_SUPSETEQ

See Also:

Constant Field Values

UC_SETUNION

public static final char UC_SETUNION

See Also:

Constant Field Values

UC_SETINTERSECTION

public static final char UC_SETINTERSECTION

See Also:

Constant Field Values

UC_LESSEQUALS

public static final char UC_LESSEQUALS

See Also:

Constant Field Values

UC_GREATEREQUALS

public static final char UC_GREATEREQUALS

See Also:

Constant Field Values

UC_ELEMENTOF

public static final char UC_ELEMENTOF

See Also:

Constant Field Values

UC_LOGICALAND

public static final char UC_LOGICALAND

See Also:

Constant Field Values

UC_LOGICALOR

public static final char UC_LOGICALOR

See Also:

Constant Field Values

keywords

public static final HashMap<String,WhileyFileLexer.Token.Kind> keywords

A map from identifier strings to the corresponding token kind.

Constructor Detail

WhileyFileLexer

public WhileyFileLexer(wyfs.lang.Path.Entry<WhileyFile> entry)
                throws IOException

Throws:

IOException

Method Detail

scan

public List<WhileyFileLexer.Token> scan()

Scan all characters from the input stream and generate a corresponding list of tokens, whilst discarding all whitespace and comments.

Returns:

scanNumericConstant

public WhileyFileLexer.Token scanNumericConstant()

Scan a numeric constant. That is a sequence of digits which gives either an integer constant, or a real constant (if it includes a dot) or a byte (if it ends in a 'b').

Returns:

scanCharacterConstant

public WhileyFileLexer.Token scanCharacterConstant()

Scan a character constant, such as e.g. 'c'. Observe that care must be taken to properly handle escape codes. For example, '\n' is a single character constant which is made up from two characters in the input string.

Returns:

scanStringConstant

public WhileyFileLexer.Token scanStringConstant()

isOperatorStart

public boolean isOperatorStart(char c)

scanOperator

public WhileyFileLexer.Token scanOperator()

scanIdentifier

public WhileyFileLexer.Token scanIdentifier()

scanWhiteSpace

public void scanWhiteSpace(List<WhileyFileLexer.Token> tokens)

scanIndent

public WhileyFileLexer.Token scanIndent()

Scan one or more spaces or tab characters, combining them to form an "indent".

Returns:

scanLineComment

public WhileyFileLexer.Token scanLineComment()

scanBlockComment

public WhileyFileLexer.Token scanBlockComment()

skipWhitespace

public void skipWhitespace(List<WhileyFileLexer.Token> tokens)

Skip over any whitespace at the current index position in the input string.

Parameters:

tokens -