Intelligent Reduction of Noise in Big Data with Report Based Filtering Technique
Publication Date: 2014-Jul-25
The IP.com Prior Art Database
1. ABSTRACT Today, Big Data innovation is running up against some formidable challenges: unchecked growth in data volumes leading to storage cost overruns, the immaturity and complexity of big data platforms, and the need to get insights from all the data, much faster. Storage costs are increasing for companies engaging in Big Data Analytics initiatives. Even though the cost of storage hardware has been declining year-over-year, those declines are still not keeping pace with data growth. Today there are several ways to solve this storage space problem, some companies may choose to throw all that data on low-cost tape, some may choose a advanced data compression technique to make sure more data can be stored with less space, and some may choose to prune the old data and keep only those relevant data to manage space. On way to reduce the storage cost of big data is to reduce / mitigate the noise. From lot of studies, it’s evident that in most of the cases the signal-to-noise ratio is very low in big data. This means that, most of the data are noise (irrelevant data), and only a tiny fraction is the signal (relevant data). Though there are several ways to solve the big data storage space problem, there is no proven techniques that can efficiently reduce or mitigate the big data noise.