| By Tony Shan |
Article Rating: |
|
| November 11, 2012 11:11 AM EST |
Reads: |
911 |
Now the election in US is over. What differs the most from the last presidential election in 2008 is the impacts of new technologies such as blogs and social media. Interestingly, Nate Silver made a surprisingly accurate prediction of the election results in his famous FiveThirtyEight blog. His near-perfect forecast solidifies the relevance and significance of big data solutions. In my opinion, 4 key factors are the critical enablers to unlock the value of big data: Modeling, Algorithm, Statistics, and Semantics (MASS).
- Modeling: First and foremost, a good model must be established to represent, capture, and ingest the large amount of data in a structured, unstructured, or semi-structured format. The nature of the data elements is a largely deciding factor for an appropriate data store.
- Algorithm: A sophisticated algorithm has to be developed to process the data in an optimized way. An easy-to-use coding method is needed to balance the local processing and global computation in a distributed fashion. For example, historical data can be tapped for generating valuable recommendations based on a user profile by means of the click-through rate and interest match metrics.
- Statistics: Statistical data analysis is becoming increasingly important, and open source packages like R make data mining more transparent. Growing commercial supports for R from the major vendors fuel the adoption, integration, and distribution of R.
- Semantics: Context-awareness is a must. Simple analysis is no longer sufficient for today's business. Complex analytics requires advanced techniques such as patterns and probabilistic reasoning. Vagueness is inevitable and got be dealt with properly to extract insights from massive data in a fuzzy way.
For more information, please contact Tony Shan (
TonyShan@live.com).
Tony Shan works as a senior consultant/advisor at a global applications and infrastructure solutions firm helping clients realize the greatest value from their IT. Shan is a renowned thought leader and technology visionary with a number of years of field experience and guru-level expertise on cloud computing, Big Data, Hadoop, NoSQL, social, mobile, SOA, BI, technology strategy, IT roadmapping, systems design, architecture engineering, portfolio rationalization, product development, asset management, strategic planning, process standardization, and Web 2.0. He has directed the lifecycle R&D and buildout of large-scale award-winning distributed systems on diverse platforms in Fortune 100 companies and public sector like IBM, Bank of America, Wells Fargo, Cisco, Honeywell, Abbott, etc.
Shan is an inventive expert with a proven track record of influential innovations such as Cloud Engineering. He has authored dozens of top-notch technical papers on next-generation technologies and over ten books that won multiple awards. He is a frequent keynote speaker and Chair/Panel/Advisor/Judge/Organizing Committee in prominent conferences/workshops, an editor/editorial advisory board member of IT research journals/books, and a founder of several user groups, forums, and centers of excellence (CoE).
Subscribe to Open Source Magazine Email News Alerts
Subscribe via RSS
