双向过滤的字符串相似连接验证方法-计算机工程与应用.PDFVIP

72 2017 ，53（9 ） Computer Engineering and Applications 计算机工程与应用 ⦾大数据与云计算⦾ 双向过滤的字符串相似连接验证方法 黄 樱，宋春花，牛保宁 HUANG Ying, SONG Chunhua, NIU Baoning 太原理工大学 计算机学院，太原 030024 School of Computer Science and Technology, Taiyuan University of Technology, Taiyuan 030024, China HUANG Ying, SONG Chunhua, NIU Baoning. Verification method for string similarity joins based on bi-directional filtering. Computer Engineering and Applications, 2017, 53 （9 ）：72-79. Abstract: A string similarity join finds similar string pairs from two sets of strings. It plays an important role in many real-world applications. Various algorithms have been proposed to address its efficiency issues. Partition-based filter-veri- fication methods, such as Pass-Join, are promising, which quickly screens out possible similar string pairs （candidate set ）by searching partitioned parts of a string in another string, in order of increasing length, and then performs similarity verification based on edit-distance. Motivated by the fact that the effect produced by filtering in the descending order of string length is better than in the ascending order, a novel bi-directional filtering-verification mechanism is proposed. At the filtering stage, it pipelines the results from length descending filtering to length ascending filtering to further reduce the size of the candidate set. At the verification stage, it makes use of the two pairs of matched substrings from the bi-directional filtering to partition the target string pairs into several short substring pairs to accelerate the verification process. Experi- mental results show that the proposed bi-directional filtering-verification algorithm outperforms the origin algorithm on real-world datasets. Key words: string similarity joins; bi-directional filtering-verification mechanism; filter-verification framework 摘 要：字符串相似连接是指在字符串集合中找出相似的字符串对，是许多应用的关键操作，寻找高效的字符串相 似连接算法已成为研究热点。基于划分的过滤-验证方法（Pass-Join ）与其他方法相比具有较高的效率。它按照字符 串长度递增的顺序访问字符串集合，通过查