![\"BI](\"http:\/\/www.saama.com\/Portals\/94553\/images\/memoryprices.jpg\")
<\/p>\nWhile most of big data is geared towards social media and stream analytics, traditional EDW can also best leverage the power of Big Data. The concept of Big Data is not new, banks have been doing it for a while using mainframe size computers. The reason it\u2019s being talked so much now is that for the first time, cheap and massive computing power and even cheaper memory has put mainframe size power in the hands of every organization, right at the time when organizations have been struggling to justify the ROI in processing such exponential data volume.<\/p>\n
<\/p>\n
While most of big data is geared towards social media and stream analytics, traditional EDW can also best leverage the power of Big Data. The concept of Big Data is not new, banks have been doing it for a while using mainframe size computers. The reason it\u2019s being talked so much now is that for the first time, cheap and massive computing power and even cheaper memory has put mainframe size power in the hands of every organization, right at the time when organizations have been struggling to justify the ROI in processing such exponential data volume.<\/p>\n
<\/p>\n
Big Data is not a performance engine. i.e. it is not a traditional database that can run queries faster. It will also not replace traditional reporting strategies. What it can do is, it can batch process millions and billions of records both unstructured and structured much faster and cheaper. What has also become possible with BigData Analytics is the ability to merge all analysis into one platform. As a direct result, data analysis has become more accurate, well-rounded, reliable and focused on a specific business capability\/advantage.<\/p>\n
Before investing money in buying commodity hardware and calling consultants to wave the big data magic wands, companies should do a lot of soul-searching because once you set the wheels in motion, it is likely to take up lot of your organization\u2019s focus. To decide where you are in the BigData spectrum it is important to look at the 4 V\u2019s \u2013 Volume, Velocity, Variety and Variability of your data as shown in the info-graphic below. <\/p>\n
![\"Big](\"http:\/\/www.saama.com\/Portals\/94553\/images\/4vs.jpg\")
<\/p>\n
![\"BigData,](\"http:\/\/www.saama.com\/Portals\/94553\/images\/4vslegend1.jpg\")
<\/p>\n
A key question to ask would be, if you have enough data volumes at the source to justify the use of Big Data processing (Average Data set > 300GB). If you don\u2019t, you should consider investing in building a traditional enterprise data warehouse and fine tuning your reporting metrics. If yes, you should move on to the next question of how you want to process this amount of data.<\/p>\n
One of the key technologies that is widely being accepted by large Enterprises for BigData Processing is Hadoop. While this technology provides the processing power, the algorithms to make sense of this data will still need to be developed in-house. The most frequent application for Hadoop is to support the \u201cTransform\u201d in traditional ETL (Extract, Transform, Load), where the data is in myriad of unstructured, semi-structured, and structured formats and loaded into terabyte-scale analytical data marts where predictive modelers and other data scientists can work their magic.<\/p>\n
Hadoop and traditional EDW technologies can co-exist in the same ecosystem as shown below. Each has its own strengths and when combined provides a potent mix for your analytical needs that we have seen in few large companies.<\/p>\n
![\"\"](\"http:\/\/enterprisebi.files.wordpress.com\/2012\/02\/hadoop-edw1.png\" "\\"Hadoop-EDW\\"")
<\/a><\/p>\n