基于PDC编码的中文文本压缩算法.pdfVIP

Computer Engineering and Applications 计算机工程与应用 2015 ，51（17） 205 基于PDC 编码的中文文本压缩算法 曾党泉 ZENG Dangquan 厦门大学嘉庚学院 信息科学与技术学院，福建 漳州 363105 School of Information Science Technology, Xiamen University Tan Kah Kee College, Zhangzhou, Fujian 363105, China ZENG Dangquan. Chinese text compression algorithm based on PDC coding. Computer Engineering and Applica- tions, 2015, 51（17）：205-209. Abstract ：According to the characteristics of Chinese text structures and the disadvantages of traditional compression algo- rithm for Chinese text compression, it proposes and implements a Chinese text compression algorithm based on PDC cod- ing. The algorithm uses dictionary compression. According to the words ’probability that appears in the Chinese text, the prefix encoded variable-length coding uses Huffman coding, it defines the depth of the phrases and short sentences that prefixed by the word, the algorithm encodes partial fixed-length coding for the phrases and short sentences which have the same prefix and depth, it constructs a compression dictionary. By comparing with the tradition compression algorithm LZW and LZSS that in the same texts, the compression algorithm ’s compression ratio increases 2.53%~40.48%, which means the compression algorithm has a better compression effect than the traditional compression algorithm. Key words ：Chinese text; compression algorithm; prefix; depth; coding; compression ratio 摘 要：针对中文文本结构的特点以及传统压缩算法对中文文本压缩的不足，提出并实现了一个基于PDC 编码的中 文文本压缩算法。该算法采用的是字典压缩方式。根据单个汉字在中文文本出现的概率，采用Huffman 编码方式 进行前缀变长编码；定义由某个汉字为前缀的词组和短语的深度；对具有相同前缀和相同深度的词组和短语进行局 部的定长编码，构成一部压缩编码字典。通过对相同文本分别使用该算法和传统的LZW 和LZSS 编码算法压缩后 得到的数据结果对比，压缩率有2.53%~40.48%的提高，表明该压缩算法有较好的压缩效果。 关键词：中文文本；压缩算法；前缀；深度；编码；压缩率 文献标志码：A 中图分类号：TP391 doi ：10.3778/j.issn.1002-8331.1411-0326 1 引言 语义信息，严重地损害了压缩率。有鉴于此，需要针对 文本压缩是数据压缩的一个分支，属于无损压缩。 中文语言的结构，充分考虑其在编码、语义方面的特点， 它的目标是通过