Migration of a database for gas storage management from Microsoft SQL Server to Snowflake on Azure
Project duration: 3 months
Brief description
For the reporting of the gas storage capacity, a global power plant operator uses MS SQL Server as database. In order to improve the application performance and user acceptance, the database is migrated to Snowflake.
Supplement
The MS SQL Server hosted on Azure is replaced by the Snowflake database on Azure. The stored procedures in the MS SQL Server are replaced by Python scripts, which are running on an Azure Batch Account. The ETL tool Talend is used to move data from an external database system to the gas storage database. Furthermore, Talend starts the now the Python scripts instead of the stored procedures on the MS SQL Server. The Talend jobs are starting the Python scripts via an Azure Batch Account. The Python scripts are mainly executing SQL statements for data transformation on Snowflake. The existing reports in Tableau and Power BI are adjusted to use the prepared clean data on Snowflake as data source.
Subject description
The customer is using a central application for the capacity management of its gas storages. The application is importing technical and business data from external systems. It calculates the free and occupied capacities of the gas storages and uses these values for midterm and longterm capacity renting. Additionally, the free capacity for intraday trading is calculated.