N-GRAM INDEKSLARIDAN FOYDALANADIGAN IMLO TUZATISH ALGORITMI
Keywords:
edit distance; n-gram; trigram; string search; Levenshtein distance.Abstract
The automatic spelling correction feature is widely used in various systems. Its main task is to identify and correct spelling errors in text, thereby saving users' time. However, when the dictionary contains millions of entries, finding the most relevant match can be time-consuming. To improve the efficiency of automatic correction, various solutions have been proposed. All of these approaches operate based on appropriate data structures and certain hypotheses to solve the problem. In this study, we explore the effectiveness of a new hybrid algorithm for spelling correction that leverages an N-gram language model to filter uncertain entries in the dictionary. This approach ensures accuracy in the dictionary while providing high efficiency in correcting errors. Experiments show that the proposed algorithm is highly effective in delivering fast and accurate results.
References
1. Jong Yong Kim and John Shawe-Taylor. ``Fast string matching using an n-gram algorithm.'' Software-Practice and Experience 24(1):79-88. January 1994.
2. Klaus U. Schulz and Stoyan Mihov “Fast string correction with Levenshtein automata” INTERNATIONAL JOURNAL ON DOCUMENT ANALYSIS AND RECOGNITION, Volume 5, Number 1, 67-85, DOI: 10.1007/s10032-002-0082- 8,2003.
3. V. Hodge and J. Austin, “A Comparison of a Novel Spell Checker and Standard Spell Checking Algorithms Pattern Recognition”, vol. 15 no. 5, pp. 1073-1081, 2003.
4. Zhao, Zuo-Peng, Yin, Zhi-Min, Wang, Qian-Pin, Xu, Xin-Zheng, Jiang, Hai-Feng and Jisuanji Yingyong “An improved algorithm of Levenshtein Distance and its application in data processing” Journal of Computer Applications. Vol. 29, no. 2, pp. 424-426. Feb. 2009 indepandant.
5. Botir Elov, Alayev Ruhillo, Abdulla Abdullayev, Narzillo Aloyev. “Yuqori N-gram modellarini o‘zbek tili matnlariga qo‘llash” Raqamli Transformatsiya va Sun’iy Intellekt ilmiy jurnali. vol. 2, pp.152-162, OCTOBER 2024.
