Perform an intelligent detection based on clusterLanguages()
WARNING: this method is EXPERIMENTAL. It is not recommended for common use, and it may disappear or its functionality may change in future releases without notice.
This compares the sample text to top the top level of clusters. If the sample is similar to the cluster it will drop down and compare it to the languages in the cluster, and so on until it hits a leaf node.
this should find the language in considerably fewer compares (the equivalent of a binary search), however clusterLanguages() is costly and the loss of accuracy from this technique is significant.
This method may need to be 'fuzzier' in order to become more accurate.
This function could be more useful if the universe of possible languages was very large, however in such cases some method of Bayesian inference might be more helpful.
Detects the closeness of a sample of text to the known languages
Calculates the statistical difference between the text and the trigrams for each language, normalizes the score then returns results for all languages in sorted order
If perl compatible, the score is 300-0, 0 being most similar. Otherwise, it's 0-1 with 1 being most similar.
The $sample text should be at least a few sentences in length; should be ascii-7 or utf8 encoded, if another and the mbstring extension is present it will try to detect and convert. However, experience has shown that mb_detect_encoding() *does not work very well* with at least some types of encoding.
Return: sorted array of language scores, blank array if no useable text was found
a sample of text to compare.
if specified, return an array of the most likely $limit languages and their scores.
Returns an array containing the most similar language and a confidence rating
Confidence is a simple measure calculated from the similarity score minus the similarity score from the next most similar language divided by the highest possible score. Languages that have closely related cousins (e.g. Norwegian and Danish) should generally have lower confidence scores.
The similarity score answers the question "How likely is the text the returned language regardless of the other languages considered?" The confidence score is one way of answering the question "how likely is the text the detected language relative to the rest of the language model set?"
To see how similar languages are a priori, see languageSimilarity()
Return: most similar language, score and confidence rating or null if no language is similar
Calculate the similarities between the language models
Use this function to see how similar languages are to each other.
If passed 2 language names, will return just those languages compared. If passed 1 language name, will return that language compared to all others. If passed none, will return an array of every language model compared to every other one.
Return: scores of every language compared or the score of just the provided languages or null if one of the supplied languages does not exist
Should speed up most detections if turned on (detault is on). In some circumstances it may be slower, such as for large text samples (> 10K) in languages that use latin scripts. In other cases it should speed up detection noticeably.