Identification of Duplicate Chunks Using Content Approach - Kaur, Gagandeep; Singh Devgan, Mandeep (International Journal of Computer Sciences and Engineering, 2025)
In this article the implementation of the functions for identification of duplicate chunks based on block, file and content approach have been discussed. The main core of the Deduplication algorithms is chunking and hashing functions. It is also referred as Deduplication granularity. The analysis of these three methods show that the content approach for deduplication is bit slow but the accuracy is good as compared to file and block strategies. It can be seen that the content method of identifying duplicate chunks is about 0.2-0.3% slower but its accuracy is higher by 1-2 % when duplicate finding method of block and file are considered. This work is useful for building duplicate content –aware applications. Especially, when it is used for checking multiple patterns, matching paraphrased content and plagiarism. The proposed methods here can be used for inline as well in the post processing type of Deduplication and it can be extended to include the concept of background and foreground processing.