| Interface | Description |
|---|---|
| TokenChecker | |
| Tokenizer |
| Class | Description |
|---|---|
| ErrorHandlerTokenizer | |
| PrintTokenizer |
Print a logging message on every token
|
| Token | |
| TokenCheckerBase |
Do nothing checker
|
| TokenCheckerLib |
Various algorithms
|
| TokenizerFactory | Deprecated
Use
TokenizerText.create()... |
| TokenizerText |
Tokenizer for all sorts of things RDF-ish
|
| TokenizerWrapper | |
| TokenizeTextBuilder |
Builder for TokenizeText
|
| Enum | Description |
|---|---|
| StringType |
The seen form of a
TokenType.STRING |
| TokenType |
Licenced under the Apache License, Version 2.0