This article appeared on DevOps
Modernizing a data warehouse ranks among the worst imaginable tasks for IT. The end result is immensely desirable. The benefits of moving from an on-premises to a cloud data warehouse are well-documented. But database migrations require huge efforts and the stakes are high. What makes it so difficult? All applications, including ETL, BI, analytics and reporting, are designed to work with a specific legacy system. Hence, moving these applications to a different system requires extensive rewrites.
Database migrations may well be the world’s most underestimated problem. In particular, data warehouse migrations are notorious for their horrendous cost overruns. Even when organizations plan for these projects to take several years, they tend to miss even the most generous deadlines. Worse, the vast majority fail to meet their objectives.
In this article, we report on the exceptional success one major enterprise had migrating its systems. They managed to modernize their system in record time with little to no downtime at all. We’ll take a look at their journey and what sets this organization apart from their peers.
The Big Challenge
They don’t get much bigger. The system in question was long regarded as one of the largest, most complex data warehouses in EMEA. It is the backbone of a market-leading integrated container logistics and shipping company.
The company’s 165TB of highly compressed data is used to track goods and shipping vessels around the world. Uninterrupted availability 24×7 is critical for this kind of operation, of course. As the central data hub, this system connects thousands of users that execute over one million queries every day.
Migrating the workloads off an aging Teradata system to Microsoft Azure Synapse was key to modernizing the data estate. IT leadership found itself faced with the question of whether it would be possible to move without disrupting their business users.
About Mike Waas, CEO Datometry
Mike Waas founded Datometry with the vision of redefining enterprise data management. In the past, Mike held key engineering positions at Microsoft, Amazon, Greenplum, EMC, and Pivotal. He earned an M.S. in Computer Science from the University of Passau, Germany, and a Ph.D. in Computer Science from the University of Amsterdam, The Netherlands. Mike has co-authored over 35 peer-reviewed publications and has 20+ patents on data management to his name.
