Data Engineer
Current• Analyzed business requirements to compose functional and implementable technical data solutions• Identified integration impact, data flows, and data stewardship• Created new data constraints and or leveraged existing constraints for reuse• Created data dictionary, Data mapping for ETL and application support, DFD, ERD, mapping documents, metadata, DDL and DML as required• Evaluated and enhanced current data models to reflect business requirements• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks• Responsible for Interpreting data, analyze results using Data Analytics and statistical techniques and provide ongoing reports• Designed ETL Logic, control tables and involved in design• Co-ordinated with Server infrastructure team when and where needed for support of SQL cluster/DR environments• Responsible for server/performance optimizations and query tuning for the complex SQL programs• Created SSIS packages, involved in setting up infrastructure on Azure• Implementing DR solution, High Availability of databases like Always On, Database mirroring, Clustering, and log shipping• Creating maintenance plans to perform consistency checks, re-indexing, and update statistics
