Weather Forecasting using MapReduce Technique

Abstract

The term Big Data basically refers to the huge volume of data. There are various sources that produce huge amount of data which includes facebook, twitter, weather station, sensors, airlines, hospitality data, newspaper etc.  Data that is collected from different sources will have different formats such as structured, semi-structured and unstructured, hence it has become difficult to process and manage by the traditional data management methods. Since there is enormous data generation, the important thing is how to store such a huge amount of data and how to manage the data. Big Data is characterized by 3V's size (volume), complexity (variety), and rate of growth (velocity) which make them difficult to process or analyze. There are some innovative technologies to capture, store and analyze petabytes of data. Hadoop manages such huge amount of data in an efficient manner. In this paper the input data is collected from NCDC (National Climatic Data Centre) then input data is stored in HDFS (Hadoop Distributed File System), later processed using MapReduce technique.