org.codelibs.elasticsearch.langfield.detect

クラス LangDetector

java.lang.Object

org.codelibs.elasticsearch.langfield.detect.LangDetector

public class LangDetector
extends java.lang.Object

LangDetector class is to detect language from specified text. Its instance is able to be constructed via the factory class LangDetectorFactory.

After appending a target text to the LangDetector instance with append(Reader) or append(String), the detector provides the language detection results for target text via detect() or getProbabilities(). detect() method returns a single language name which has the highest probability. getProbabilities() methods returns a list of multiple languages and their probabilities.

The detector has some parameters for language detection. See setAlpha(double), setMaxTextLength(int) and setPriorMap(Map).

 import java.util.ArrayList;
 import org.codelibs.elasticsearch.langfield.detect.LangDetector;
 import org.codelibs.elasticsearch.langfield.detect.LangDetectorFactory;
 import org.codelibs.elasticsearch.langfield.detect.Language;

 class LangDetectSample {
     public void init(String profileDirectory)  {
         LangDetectorFactory.loadProfile(profileDirectory);
     }
     public String detect(String text)  {
         LangDetector detector = LangDetectorFactory.create();
         detector.append(text);
         return detector.detect();
     }
     public List<Language> detectLangs(String text)  {
         LangDetector detector = LangDetectorFactory.create();
         detector.append(text);
         return detector.getProbabilities();
     }
 }
 

• 4x faster improvement based on Elmer Garduno's code. Thanks!

作成者:

Nakatani Shuyo, shinsuke

関連項目:

LangDetectorFactory

フィールドのサマリー

フィールド

修飾子とタイプ	フィールドと説明
static java.lang.String	UNKNOWN_LANG

コンストラクタのサマリー

コンストラクタ

コンストラクタと説明
LangDetector(LangDetectorFactory factory) Constructor.

メソッドのサマリー

修飾子とタイプ	メソッドと説明
void	append(java.io.Reader reader) Append the target text for language detection.
void	append(java.lang.String text) Append the target text for language detection.
java.lang.String	detect() Detect language of the target text and return the language name which has the highest probability.
java.util.List<Language>	getProbabilities() Get language candidates which have high probabilities
void	setAlpha(double alpha) Set smoothing parameter.
void	setMaxTextLength(int maxTextLength) Specify max size of target text to use for language detection.
void	setPriorMap(java.util.Map<java.lang.String,java.lang.Double> priorMap) Set prior information about language probabilities.
void	setVerbose() Set Verbose Mode(use for debug).

クラスから継承されたメソッド java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

フィールドの詳細

UNKNOWN_LANG

public static final java.lang.String UNKNOWN_LANG

関連項目:

定数フィールド値

コンストラクタの詳細

LangDetector

public LangDetector(LangDetectorFactory factory)

Constructor. LangDetector instance can be constructed via LangDetectorFactory.getLangDetector().

パラメータ:

factory - LangDetectorFactory instance (only LangDetectorFactory inside)

メソッドの詳細

setVerbose

public void setVerbose()

Set Verbose Mode(use for debug).

setAlpha

public void setAlpha(double alpha)

Set smoothing parameter. The default value is 0.5(i.e. Expected Likelihood Estimate).

パラメータ:

alpha - the smoothing parameter

setPriorMap

public void setPriorMap(java.util.Map<java.lang.String,java.lang.Double> priorMap)

Set prior information about language probabilities.

パラメータ:

priorMap - the priorMap to set

setMaxTextLength

public void setMaxTextLength(int maxTextLength)

Specify max size of target text to use for language detection. The default value is 10000(10KB).

パラメータ:

maxTextLength - the maxTextLength to set

append

public void append(java.io.Reader reader)
            throws java.io.IOException

Append the target text for language detection. This method read the text from specified input reader. If the total size of target text exceeds the limit size specified by setMaxTextLength(int), the rest is cut down.

パラメータ:

reader - the input reader (BufferedReader as usual)

例外:

java.io.IOException - Can't read the reader.

append

public void append(java.lang.String text)

Append the target text for language detection. If the total size of target text exceeds the limit size specified by setMaxTextLength(int), the rest is cut down.

パラメータ:

text - the target text to append

detect

public java.lang.String detect()

Detect language of the target text and return the language name which has the highest probability.

戻り値:

detected language name which has most probability. code = ErrorCode.CantDetectError : Can't detect because of no valid features in text

getProbabilities

public java.util.List<Language> getProbabilities()

Get language candidates which have high probabilities

戻り値:

possible languages list (whose probabilities are over PROB_THRESHOLD, ordered by probabilities descendently code = ErrorCode.CantDetectError : Can't detect because of no valid features in text