Objective: Optimizing performance and cost-effectiveness
A US Based healthcare service company wanted to decommission its existing Netezza environment and leverage cloud-native options to modernize, improve workload performance, remove maintenance overhead, and gain scalability.
Challenges: Obsolete legacy environment failing to meet business needs
There were several challenges with the client’s legacy on-premise data warehouse, including poor performance and the inability to handle high volumes of data. In some cases, they had long-running ETL batches for nearly 10-12 hours in a given day, impacting the downstream process. They constantly faced high CPU utilization on both Netezza and DataStage servers. To perform load balancing, they used two DataStage servers, escalating cost and maintenance. The client sought a partner to help them organize and store their ever-growing data in a secure, scalable, and cost-effective manner to meet their business needs.
Solutions: Leveraging cloud-native solutions
- Source systems; Primarily DAAC and CERNER
- Datametica provided a detailed analysis of the existing Netezza platform using Datametica Eagle, which is an automated assessment & data migration planning technology
- One-time historical data migration from Netezza to GCP
- Conversion/Rewrite of DataStage Master sequences, NZ SQL, NZ stored procedures, UDFs, and other transformation scripts to GCP native using Datametica’s Automated code (SQL, Script) and ETL conversion technology service – Raven.
- Reusable data frames were built to perform data cleansing activity and to remove special characters.
- Setup orchestration and scheduling of workloads using GCP Cloud Composer.
- Establish connectivity with Tableau/Cognos, and assist the client team with Report repointing activities.
- Auto data validation using Datametica’s Automated data validation tool – Pelican technology, between Netezza and Google Cloud Platform.
Volumetrics:
| Type | Parallel Jobs | Stored Procedures | Tables | Views |
| Count | ~3,000 | ~2,000 | 8,800 | 1,500 |
Client Benefits: Enhanced performance and scalability
- Faster migration to GCP using Datametica’s automated data migration tools.
- Providing access to Google technologies, best practices, and recommendations
- Enhancement of operational capabilities and reporting capabilities
- Managing resources and security on GCP with access control using IAM.
GCP Products Used
Datametica Products Used
