Next Generation Analytics & Innovation Platform (Stock Exchange)

advancedanalytics

Client

7th largest stock exchange

Our client is the seventh largest stock exchange in the world by market capitalization, with more than 5,000 companies listed on it. The size and scale of the company’s operations posed the following challenges:

  • Exponential data growth
  • An inability to carry out complex analytics quickly
  • Information layer scattered across multiple systems, making consolidation pertinent

Solution

Migrating to Hadoop & 'R'

Datametica implemented the project in various phases:

  • Phase 1 - Commission a Big Data Platform and consolidate terabytes of data into an Enterprise Data Hub (Go Live Nov 2013)
  • Phase 2 – Migrating of all the reporting and ad hoc analysis to Hadoop using an open source reporting platform
  • Phase 3 – Migrating Predictive Modeling for Fraud Detection to ‘R’

Deliveries

The main aspect of the solution consisted of a 150-terabyte Hadoop Cluster to move the traditional information management platform to an open source one. Datametica proposed and delivered the following:

  • Create an Enterprise Data Hub and consolidate data silos
  • Migrate Reporting and Analytics to this framework using open source technologies

Deliveries

  • Migrate the statistical modeling framework from proprietary technology to Open Source R
  • Create a comprehensive CMS (Content Management System) to replace the proprietary legacy system
  • Use this cluster as an intelligent archive and do away with expensive storage such as emails, documents, backups, and archives

Benefits

costsavings
Cost saving

Savings amounted to $150K in annual software maintenance and $1M in future investment to maintain the projected growth rate for the next three years

customer360
Customer experience

Seamless transition to the new data platform was implemented so efficiently that end-users did not experience a single glitch

operational
Performance

The solution gave our client what it wanted – a highly scalable environment. Our client experienced at least 4x improvement in query performance in comparison to the earlier environment. It is now possible to run historical queries spanning many years using very complex algorithms

operational
Operational efficiency

With the introduction of algorithm trading in the Indian stock markets, the trade to order ratio has gone up 125 times. Our client got the NetApp-BW|Businessworld Innovation Award in the Big Data Innovation category in Mar-2014

magic
Tools / Technologies
  • Hadoop Ecosystem
  • Pig
  • Hive
  • Impala
  • R
Top