Solution

Based on client interaction, the product requirements were studied and the long term and short-term road-maps were cross referenced to the product functionality and technology stack. The architecture team worked on picking the right Amazon AWS components necessary for efficient and cost effective functionality. A decision was made to build the entire stack on a server-less architecture using AWS Lambda as the base computation layer.

Data stream models for various data points such as operating system vitals, database parameters, JVM heap and memory were designed. Data streams were queued, pre-processed and stored against a combination of AWS RDS and AWS DynamoDB systems.

A robust visualization framework, based on Angular & D3.js, was built for log analysis. Statistical models were built and deployed for anomaly detection and results cascaded to an alerting queue.