Functional Extension of a database for gas storage management on Snowflake on Azure

Some data sources periodically save XML files on an Azure Blob Storage. Azure Function Apps (written in C#) are triggered every time a new file is saved on the Blob Storage and process the data in the XML files in order to save it in the Snowflake database. For further processing Python scripts on Azure Batch Accounts are used. The ETL tool Talend is used to start the Azure Batch Accounts, which are calling these Python scripts. Furthermore, Talend is used to query some other data sources which provide API endpoints. The data is then also directly saved on Snowflake and further processed by the Python scripts. The Python scripts are mainly executing SQL statements for data transformation on Snowflake. The existing reports in Tableau and Power BI are adjusted to use the prepared clean data on Snowflake as data source.