Big Data Whiteboard: sqrrl Lessons Learned from the NSA
Adam Fuchs, CTO and Co-Founder of sqrrl, does a whiteboard session with Wikibon's Dave Vellante on best practices in Big Data application development. One of my visionary colleagues at EMC, a leader in digital marketing always ends his presentations with the bold sentence "Let's Get Started". So when I was asked to develop the storyboard to demonstrate the specific benefits and applications of Big Data Analytics for clients in capital markets, hedge funds and asset and wealth management, I immediately borrowed this line. We had already spent enough time with theoretical discussions, idea generation and brainstorming workshops. It was time to put the pedal to the metal and demonstrate real results. We are starting to come out of one the worst credit crises in recent history. Financial firms have learned the lessons of the limitations of incomplete data and piece-meal models for managing portfolios and decision making and become more sensitive to their real time needs. Information driven companies want to harness Big Data Analytics to play pivotal roles to optimize their use of capital and manage their risks. During this year's Gartner Business Intelligence & Analytics Summit, the key message from Nate Silver, the analytics expert and New York Times columnist was about the need for a practical strategy and specific application for Big Data Analytics. His keynote speech was applauded by a packed audience of more than 1,400 business and IT professionals when he said "Whatever you do, don't give into the big data hype." All complex problems and programs involve a learning curve and tackling Big Data is no exception. Some financial firms are re-thinking existing models and risk management analytics fuelled by readily available, open source Big Data technologies. But they should be cautious of the hidden challenges they pose. Most of these "do-it-ourselves" Big Data Analytics sandboxes (like those on Amazon Cloud) seem to take a "trial and error" approach. To help clients move beyond random explorations and more clearly move into the business realities with Big Data, we started with one solid Big Data Analytics business use case that has common implication and that is a crucial part of the investment process at both the buy side and the sell-side firms.