Big data is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making. Encompassing everything from click stream data from the web to genomic and proteomic data, it’s a heterogeneous mix of data both structured (traditional datasets –in rows and columns like DBMS tables, CSV’s and XLS’s) and unstructured data like e-mail attachments, manuals, images, PDF documents, medical records such as x-rays, ECG and MRI images, forms, rich media like graphics, video and audio, contacts, forms and documents.Businesses are primarily concerned with managing unstructured data, because over 80 percent of enterprise data is unstructured and require significant storage space and effort to manage.
What is Big Data?
“Big data” refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. Google’s technical response to the challenges of Web-scale data management and analysis was simple, by database standards, but kicked off what has become the modern “Big Data” revolution in the systems world . Google File System (GFS) was created to handle the challenge of Web-scale storage, providing clients with the familiar Olive byte-stream abstraction for extremely large files whose content can span hundreds of machines in clusters created using inexpensive commodity hardware. This helps the processing of data in such large files.
Big data has arisen because digitization has accelerated the growth in data across every organization, industry and economy. Each day, posts are made on Facebook, tweets sent on Twitter and videos uploaded to YouTube. Networked sensors collect huge amounts of data from mobile phones, gas and electrical meters, aircraft engines, drilling platforms and atmospheric equipment. Satellites gather weather and geographical data and military intelligence. Data is created as by-products from everyday activities such as the transaction data collected by supermarket tills.
According to IDC, the world’s digital output soared from 180 exabytes in 2006 to about 1,800 exabytes in 2011. (One exabyte equals one billion gigabytes.) Data volume is expected to reach 35,000 exabytes by 2020, representing a 20-fold increase in the next 10 years. To understand how big data affects companies, it helps to look at three key characteristics such as:
- Information Strategy
- Data Analytics
- Enterprise Information Management.
Future of Big Data
Big data is seen as a huge potential driver of returns for the next generation of multi-billion-dollar software companies. It represents the fastest-growing market for information systems globally. Analysts believe that the total addressable market is worth US$100 billion (A$102 billion).
Today, various startup have invested heavily in global data centers designed to interpret big data and they are scouting for the best talent. IBM, Microsoft, Oracle and SAP have spent more than US$15 billion acquiring software companies that specialize in business intelligence tools as they look for the best software developers, statisticians, quant analysts, econometricians and data analysts.
Based on these facts and analysis part there is one survey which was being conducted this year by the firm Information week.
It was observed that various parameters and conditions are kept constant and varying for the gathering and collecting the information, finally we got an outcome of survey as per our expectations and also observed that this survey is fruitful and sensible for business people up to a certain extent. The survey is in the form of pictograph which is released by the company itself.
The “Big Data Market: 2014 – 2020 – Opportunities, Challenges, Strategies, Industry Verticals & Forecasts” is a in-depth assessment of the Big Data ecosystem including key market drivers, challenges, investment potential, vertical market opportunities and use cases, future roadmap, value chain, case studies on Big Data analytics, vendor market share and strategies. Not only this, the future of this big data has bi fold increased in which Hadoop plays a huge role.
We are still on our way to determine what Hadoop is and how it is coming in the picture under big data. In the upcoming part we would cover all the aspects of Hadoop and why it has become an essential part of Big Data.