A Static Dictionary-Based Approach To Compressing Short Texts Kisa Metinleri Sikiçtirmak Için Statik Sözlük Tabanli Bir Yaklaçim
6th International Conference on Computer Science and Engineering, UBMK 2021, Ankara, Türkiye, 15 - 17 Eylül 2021, ss.342-347, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/ubmk52708.2021.9559035
- Basıldığı Şehir: Ankara
- Basıldığı Ülke: Türkiye
- Sayfa Sayıları: ss.342-347
- Anahtar Kelimeler: K-Means, Machine learning, SDC, Short Text Compression, Static Dictionary Compression
- Kırklareli Üniversitesi Adresli: Evet
Özet
In this study, Static Dictionary Compression (SDC) method, which is an approach developed to compress short texts, is proposed. The word-based static dictionaries used in this approach were obtained from clusters formed as a result of running a clustering method repeatedly until certain criteria are met. Short text is compressed with the dictionary that has the largest number of words in common with it. It has been shown by tests conducted with datasets containing short texts in 6 different languages that the proposed method compresses better than the general purpose compression methods Gzip, Bzip2, Zstd and PPMd. In the tests made with the data set containing only English short texts, it has been shown that the SDC method can compress better than the smza, shoco and b64pack methods used to compress short texts, and Brotli, which gives good results in short texts because it uses a static dictionary.