11-11-2014, 10:16 AM
Abstracts: Every day, we create 2.5 quintillion bytes of data — so much that 90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals to name a few. This types of data generated are known as Big Data. Now a day every company is in a race to compete in global market for that they need accurate results of their real time data in order to take beneficial decisions this leads to next-gen data analytics for building a flexible and efficient statistics. Google Published White papers on GFS and Big table for processing and handling big data using commodity hardware. Inspired from GFS and Big table Apache community created open source framework Hadoop with its File System HDFS and similar to Big table they have created HBase. Hadoop is a framework that supports data-intensive distributed applications and is used to process data on thousands of computers. HBase is an open source distributed Key/Value store based on the idea of Big Table. It is being used in many data-centre applications (e.g. Facebook, Twitter, etc.) because of its portability and massive scalability. For this kind of system, low latency and high throughput is expected when supporting services for large scale concurrent accesses. The Company for which we are doing project has several home automation system and sensor devices, these devices are generating large amount of unstructured data that cannot be handled using relational database systems, so in order to handle such type of data we are using Hive and Hadoop. We will be generating several useful statistics based on the company needs that will be useful to monitor current performance and will be beneficial for their future plans and strategies.