Uses of Interface
org.canova.nlp.tokenization.tokenizer.Tokenizer

Packages that use Tokenizer

Package	Description
org.canova.nlp.tokenization.tokenizer
org.canova.nlp.tokenization.tokenizerfactory
org.canova.nlp.vectorizer

Uses of Tokenizer in org.canova.nlp.tokenization.tokenizer

Classes in org.canova.nlp.tokenization.tokenizer that implement Tokenizer

Modifier and Type	Class and Description
class	DefaultStreamTokenizer Tokenizer based on the StreamTokenizer
class	DefaultTokenizer Default tokenizer
class	PosUimaTokenizer Filter by part of speech tag.
class	UimaTokenizer Tokenizer based on the passed in analysis engine

Uses of Tokenizer in org.canova.nlp.tokenization.tokenizerfactory

Methods in org.canova.nlp.tokenization.tokenizerfactory that return Tokenizer

Modifier and Type	Method and Description
Tokenizer	PosUimaTokenizerFactory.create(InputStream toTokenize)
Tokenizer	UimaTokenizerFactory.create(InputStream toTokenize)
Tokenizer	TokenizerFactory.create(InputStream toTokenize) Create a tokenizer based on an input stream
Tokenizer	DefaultTokenizerFactory.create(InputStream toTokenize)
Tokenizer	PosUimaTokenizerFactory.create(String toTokenize)
Tokenizer	UimaTokenizerFactory.create(String toTokenize)
Tokenizer	TokenizerFactory.create(String toTokenize) The tokenizer to createComplex
Tokenizer	DefaultTokenizerFactory.create(String toTokenize)

Uses of Tokenizer in org.canova.nlp.vectorizer

Methods in org.canova.nlp.vectorizer with parameters of type Tokenizer

Modifier and Type	Method and Description
abstract void	TextVectorizer.doWithTokens(Tokenizer tokenizer) Increment counts, add to collection,...
void	TfidfVectorizer.doWithTokens(Tokenizer tokenizer)