中圖分類號： TP391
文獻標識碼： A
DOI： 10.19358/j.issn.2096-5133.2022.04.011
引用格式： 侯開茂，韓慶敏，吳云峰，等. 一種基于Simhash算法的重復域名數(shù)據(jù)去重方法[J].信息技術(shù)與網(wǎng)絡安全，2022，41(4)：71-76.

Method for deleting duplicate domain name data based on Simhash algorithm

Abstract： With the development of digital science and technology, the amount of data that needs to be transmitted and stored in various fields has risen sharply. However, the number of repetitions in these data occupies a large proportion. This not only increases the cost of using data, but also reduces the efficiency of data processing. Domain name is a kind of data with large storage capacity and extremely high requirements for processing speed. In order to save storage cost and improve transmission efficiency, this paper proposes a method for deleting duplicate domain name data based on Simhash algorithm. Compared with the traditional data deduplication technology, this method combines the structural characteristics of the domain name data, and introduces the Simhash algorithm to design a deduplication method for the domain name data. The experimental results show that compared with the traditional data deduplication technology, this method is more efficient in deleting duplicate domain name data and has better practical application value.

Key words : data deduplication；domain name；Simhash；data block