Home » Case Study »
The Objective:
The national postal services in the United Kingdom wanted to migrate their on-premise Teradata to GCP. The objective of this engagement was to migrate the analytics, reporting and data science capabilities that are currently on the Teradata Migration to Google Cloud Platform (GCP) and leverage the cloud-native technologies for reduced operational cost and improved performance.
Challenges: Scalability and Performance issue
The client faced challenges with high operational costs and performance issues while running data on Teradata. In peak season, the existing Teradata system wasn’t able to scale which resulted in poor performance and technical issues during the critical period. Lack of SMEs with cloud expertise was another reason for associating with Datametica.
The Solution: Future Data Platform Program on GCP
- Datametica simplified the cloud migration process by offering a detailed assessment of the existing Teradata environment using Eagle- an automated assessment & data migration planning tool. Through this analysis, we provided the best possible approach of GCP migration.
- One-time historical data migration from on-premise to GCP.
- Code Conversion of Teradata SQL and ETL (Datastage and . NET) scripts to BQ using Datametica’s Automated code (SQL, Script) and ETL conversion Tool – Raven.
- Migrated ELT/ ETL Pipeline (DataStage & .NET) to Google BigQuery and Cloud compose.
- Shell Scripting and Python Scripting were used to manage ETL Pipeline.
- Designed a Framework for complex XML Data processing and loading into the GCP BigQuery.
- Performed Data Validation using Datametica’s automatic data validation tool – Pelican, for data quality assurance. This ensured incremental data loads, based on converted code during migration, match at a table, row and cell level for production parallel run between Teradata systems to Google BigQuery which gave them the confidence to decommission the Teradata environment.
- Datametica used Google Cloud Storage, Big Query, Dataflow, IAM, Pub/Sub, Stack Driver, Composer to simplify the complex data migration to google cloud and overcome challenges related to time, cost and scalability.
Client Benefits: High Scalability with 40% Improved Performance.
- 50% faster migration to GCP using Datametica’s automated data migration tools.
- Reduced cost for the migration process.
- The high scalability of GCP helped them deliver high customer service in peak seasons.
- 40% improvement in performance.
Google Products Used
Datametica Products Used
