org.molgenis.data.semanticsearch.string

Class NGramDistanceAlgorithm

java.lang.Object

org.molgenis.data.semanticsearch.string.NGramDistanceAlgorithm

public class NGramDistanceAlgorithm
extends Object

This class has implemented n-gram distance algorithm so a similarity score could be calculated between two sequences. The two input strings would be tokenized depending on what nGrams we have specified. The default ngram is 2 which can be changed in the constructor. The two groups of tokens will be further used to work out the similarity score. In addition, by default a list of stop words has been defined, in the method stringMatching(), one of the parameters "removeStopWords" indicates whether the stop words will be used to remove the useless or meaningless words from the String. This the stop words could be customized by setStopWords(List stopWords) or setStopWords(String[] stopWords). How to use? LevenShteinDistanceModel model = new LevenShteinDistanceModel(2); double similarityScore = model.stringMatching("Smoking", "Smoker", false); System.out.println(similarityScore); The other way List tokens_1 = model.createNGrams("Smoking", false); List tokens_2 = model.createNGrams("Have you smoked last year?", true); //remove stop words! double similarityScore = model.calculateScore(tokens_1, tokens_2);

Author:

Chao Pang

Field Summary

Fields

Modifier and Type	Field and Description
static Set<String>	STOPWORDSLIST

Constructor Summary

Constructors

Constructor and Description
NGramDistanceAlgorithm()

Method Summary

Modifier and Type	Method and Description
static Map<String,Integer>	createNGrams(String inputQuery, boolean removeStopWords) create n-grams tokens of the string.
static double	stringMatching(String queryOne, String queryTwo)
static double	stringMatching(String queryOne, String queryTwo, boolean removeStopWords)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STOPWORDSLIST

public static final Set<String> STOPWORDSLIST

Constructor Detail

NGramDistanceAlgorithm

public NGramDistanceAlgorithm()

Method Detail

stringMatching

public static double stringMatching(String queryOne,
                                    String queryTwo)

stringMatching

public static double stringMatching(String queryOne,
                                    String queryTwo,
                                    boolean removeStopWords)

createNGrams

public static Map<String,Integer> createNGrams(String inputQuery,
                                               boolean removeStopWords)

create n-grams tokens of the string.

Parameters:

inputQuery -

removeStopWords -

Returns:

a map of ngram tokens with the corresponding frequency