Text Mining Using Frequent Pattern Analysis and Message Passing

Abstract

Text mining is a Computer Science technique to analyze text data. Text mining is text analysis, is the process of deriving high quality information from text. Text mining is to convert text into data for suitable analysis. It allows us to investigate relationship among patterns which would otherwise be extremely difficult. Various techniques are used to mining the frequent patterns in the given text which are applicable to analyze the information in huge documents. The parallel construction of FP-Trees and parallel mining on multi cores is a popular tree projection based mining algorithm. Once each processor counts the frequency of each item using its local data partition, all worker processors send the local count to the master processor which combines them and generate global count. The parallel implementation of FP-tree may show good speedups but sending the local results to master on distributed environment and merging the patterns count on master core are overhead which consumes a considerable time. This study aims at to analyze various frequent pattern mining techniques used to extract information from texts especially on multi cores and going to adopt a new technique for finding frequent patterns, which used the Dictionary based compression algorithm(LZW). The new technique is implemented with single processor as so as with multi processor using message passing technique. The main objective of this research is enhancing the speed and reduce the memory consumption required to extract the frequent patterns form the given textual data. The parallel implementation of our proposed LZW based algorithm with three datasets Webdoc, Kosarak and Trump is compared with parallel implementation of FP-Growth on single and multi core. The results shows good performance in speedup, Latency and Efficiency in proposed LZW based algorithm.