Keyword Based Extraction of Information Using Map Reduce Method

Abstract

Three characteristics define Big Data: volume, variety, and velocity. Size (volume), complexity (variability), and rate of growth (velocity) make them difficult to capture, manage, process or analyzed. Big data is large volume, heterogeneous, distributed data. Big data can be structured, unstructured or semi-structured. Usually data is generated from various different sources. Hadoop is an open source framework that supports the processing and storage of extremely large data sets in a distributed computing environment. Hadoop makes it possible to run applications on systems with thousands of commodity hardware nodes, and to handle thousands of terabytes of data. Hadoop also includes a Distributed File System (HDFS), which manages distributed data on different node and Map-Reduce for programming paradigm. Most text files are available in pdf format as per demand. Even all research papers are available in pdf format only and extracting a text from pdf format is one of the most difficult jobs. In this paper, we use a keyword based extraction method for extracting the text from txt file and with the help of these keywords we can get all the detail on that part of the research paper or any pdf file.