Application Architects love layers; and I think it is time for a new layer – the analytics layer.
“You know what else everybody likes? Parfaits! Everybody likes a parfait.” – Donkey
Data analytics have become too compelling to ignore any longer – even at the application level; and not just within specialized applications but within the standard application software stack. Architects have always been about taming complexity and the techniques coming out of the analytics world bring powerful tools to the table. We’ve always had a data layer. It’s time for the analytics layer.
Early trends in data warehousing and BI had the unfortunate side effect of creating silos within organizations and within software applications. Analytics was a separate entity. Star schemas, denormalized data and batch processing took significant processing power and therefore was expected to be “off-loaded” from the standard operational system. Dashboards and pivot tables were bolted on as a way to see into the state. And though over time the delay was minimized to the point of “near-real-time” the silos remain. Today, analytics is not a layer; it’s a module.
But the true power of analytics lies in their ability to provide effective simplicity. Google became the most profitable company in the world not by creating a simple search, other companies had the idea of a single text entry, but by making that simple search nearly always find what the user was looking for through developing the most effective ranking algorithm. Google didn’t just build a data layer, they built a high-powered analytics layer and stacked their software on top of it.
An analytics layer holds the promise of delivering intelligence directly to the point of decision within the application. Up to now we have offered users choices but expect them to bring the intelligence to the table. We need to go beyond explaining the options available to bringing related real-time information and statistics to bear on the decisions being made.
Consider two simple examples: Amazon and Stack Overflow. Both of these websites changed the way decisions were made based on bringing highly reliable information to the point of decision. The approach is different – simple ratings vs. stack ranking – but each generated incredible gravity for their site because of the power of aggregated, targeted, reliable information delivered right at the point of decision. Both applications rely on a powerful analytics layer.
Interestingly, the analytics delivered in these examples did not require advanced algorithms, neural networks or machine learning – but rather the answer to one question: “What information would give the user the best chance to make the right decision?”
Sometimes the answer to that question is extremely simple – a basic comparison, information about what other users decided, etc. In other cases, more sophisticated algorithms and statistical methods are required. But in all cases, the question needs to be asked. Having an Analytics Layer forces the question to be asked. When designing the Data Layer, the Data Architect asks what options need to be provided to the user. When designing the Analytics Layer, the Analytics Architect asks what intelligence gives the best chance for a successful choice. And when designing the UI Layer the UX architect asks how best to bring those two things together.
Vendors and platforms are touting data analytics products and tools, but until application architects begin to think in terms of an analytics layer the true potential of these techniques will remain largely untapped.