Senior Data Engineer
Current
Pittsburgh, Pennsylvania, United States
- Built and implemented relational servers and databases for the Azure Cloud.
- Delivered and approached the data migration for advancement of current solutions from on-premises systems and apps with pipelines to Azure cloud is being developed.
- Implementing Azure Data Factory, Spark SQL, and U-SQL Azure Data Lake Storage Gen2 (ADLS) Analytics, extract, transform, and load data from sources systems to Azure Data Storage services.
- Integrated the data from on-premises SQL servers to cloud databases (Azure Synapse Analytics & Azure SQL DB).
- Created a Spark job using Azure Databricks to replace the current ETL/SSIS solution and provided Data warehouse solutions on Azure Synapse using Polybase/external tables.
- Performed with data analysis using HiveQL, HBase, and custom Map Reduce programs as well as experience importing and exporting data from RDBMS to HDFS and Hive.