org.canova.nlp.vectorizer

Class TfidfVectorizer<VECTOR_TYPE>

java.lang.Object

org.canova.nlp.vectorizer.TextVectorizer<VECTOR_TYPE>

org.canova.nlp.vectorizer.TfidfVectorizer<VECTOR_TYPE>

All Implemented Interfaces:

Vectorizer<VECTOR_TYPE>

public abstract class TfidfVectorizer<VECTOR_TYPE>
extends TextVectorizer<VECTOR_TYPE>

Tf idf vectorizer

Author:

Adam Gibson

Nested Class Summary

Nested classes/interfaces inherited from interface org.canova.api.vector.Vectorizer

Vectorizer.RecordCallBack

Field Summary

Fields inherited from class org.canova.nlp.vectorizer.TextVectorizer

cache, MIN_WORD_FREQUENCY, minWordFrequency, STOP_WORDS, stopWords, TOKENIZER, tokenizerFactory

Constructor Summary

Constructors

Constructor and Description
TfidfVectorizer()

Method Summary

Modifier and Type	Method and Description
TokenizerFactory	createTokenizerFactory(Configuration conf) Create tokenizer factory based on the configuration
abstract VECTOR_TYPE	createVector(Object[] args)
void	doWithTokens(Tokenizer tokenizer) Increment counts, add to collection,...
abstract VECTOR_TYPE	fitTransform(RecordReader reader)
abstract VECTOR_TYPE	transform(Collection<Writable> record)

Methods inherited from class org.canova.nlp.vectorizer.TextVectorizer

fit, fit, initialize, toString, wordFrequenciesForRecord

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.canova.api.vector.Vectorizer

fitTransform

Constructor Detail

TfidfVectorizer

public TfidfVectorizer()

Method Detail

doWithTokens

public void doWithTokens(Tokenizer tokenizer)

Description copied from class: TextVectorizer

Increment counts, add to collection,...

Specified by:

doWithTokens in class TextVectorizer<VECTOR_TYPE>

createTokenizerFactory

public TokenizerFactory createTokenizerFactory(Configuration conf)

Description copied from class: TextVectorizer

Create tokenizer factory based on the configuration

Specified by:

createTokenizerFactory in class TextVectorizer<VECTOR_TYPE>

Parameters:

conf - the configuration to use

Returns:

the tokenizer factory based on the configuration

createVector

public abstract VECTOR_TYPE createVector(Object[] args)

fitTransform

public abstract VECTOR_TYPE fitTransform(RecordReader reader)

transform

public abstract VECTOR_TYPE transform(Collection<Writable> record)