Muhammad Naveed Email & Phone Number

Eden Prairie , Mn, Us

• Led the design, development, and ongoing maintenance of a robust Data Warehousing and Business Intelligence system within the Azure cloud environment.• Spearheaded the creation and maintenance of a Data Lake using Azure Data Lake Storage (ADLS) Gen 2, leveraging the power of Databricks with Scala code technology. This scalable Data Lake served as the foundational repository for diverse healthcare data• Successfully orchestrated the migration of critical healthcare data, including member information, provider details, and claims data, from disparate sources, including medical records, Medicare databases, and marketplace data, into the Azure cloud.• Executed a seamless transition from an outdated QNXT system to Azure ADLS Gen 2 using Databricks, ensuring data integrity and minimizing downtime.• Monitored and optimized Cosmos DB performance using Azure Monitor and Application Insights, identifying and addressing potential bottlenecks• Leveraged Databricks for job orchestration and setup, authoring intricate scripts in Scala and, when necessary, Python, within Databricks notebooks to streamline data processing and transformation.• Implemented and configured Databricks jobs, utilizing a customized Azure autoloader utility script within Databricks notebooks, optimizing data loading processes and improving overall system efficiency.• Collaborated effectively with cross-functional teams, including data engineers, data scientists, and business analysts, to gather requirements, design data pipelines, and deliver actionable insights.• Integrated Azure services such as Azure Data Factory and Azure DevOps into the project workflow, ensuring seamless data integration, deployment, and version control.Technology- Azure Data Factory, Databricks, Azure Devops, Azure Datalake gen2, INFORMATICA Power Centre 10.2, Teradata 14.x, Oracle 19c, Oracle Exadata, SQL Developer, MSBI, MicroStrategy, WinSCP, Mainframe, Oracle 12c, MS SQL 12, SSIS, UNIX Scripting.

Long Beach, California, Us

• Processed data into HDFS by developing solutions, Analyzed the data using MapReduce, Pig, Hive and produce summary results from Hadoop to downstream systems.• Worked with data transfer from on-premises SQL servers to cloud databases Azure SQL DB.• Created Pipelines that were built in Azure Data Factory using Linked Services/ Datasets/ Data Flow/ Pipeline/ to extract, transform, & load data from a variety of sources including Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool, & reverse.• Used Airflow for scheduling and orchestrating the data pipelines.• Created Azure Blob Trigger Function using Python to load the data from ADLS to Azure SQL Server• Integrated data storage options with Spark, notably with Azure Data Lake Storage, and Blob storage.• Responsible for building scalable distributed data solutions using Hadoop.• Designed and developed complex Power BI dashboards to track key performance indicators (KPIs) and provide real-time insights for executive decision-making• Analyzed large amounts of data sets to determine the optimal way to aggregate & report on it.• Developed simple to complex Map Reduce Jobs using Hive to cleanse & load downstream data• Created partitioned tables in Hive & Managed & reviewed Hadoop log files.• Involved in creating Hive tables, loading with data & writing Hive queries, which will run internally in Map Reduce way.• Integrated Cosmos DB with Azure Functions to create serverless applications that respond to database changes and events• Utilized Power BI Embedded to integrate interactive reports within custom applications, providing seamless access to insights• Developed complex queries using SQL API, ensuring efficient data retrieval and analysis for various applications• Involved in managing and reviewing Hadoop log files.• Develop automation scripts using PySpark and pandas for test scenarios• Created Hive structure to load the data on and above created external tables.

Melksham, Wiltshire, Gb

• Involved in full project life cycle implementation (Requirements gathering to production phase).• Documented business requirements developed test cases and tracking projects through to successful implementation.• Write ETL specifications and mapping, transformations documents• Implemented performance tuning of Sources, Targets, Mappings and Sessions by identifying bottlenecks and used Debugger to debug the complex mappings and fix them.• Analyzed data model and mapping documents, work with stake holders to resolve finding gaps through query logs.• Developed data loading strategies, transformation for business to analyse the datasets. • Wrote Hive scripts to load data from one stage into another and implemented incremental load with the changed data architecture• Performed data analysis, queries on Hive and Enhanced Hive performance by implementing Optimizing and Compressing Techniques• Performed transformations on data and mapped the source data to target and load them into Hive tables• Involved in developing Spark applications using Spark-SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for analyzing and transforming the data to as per business requirement.• Extracting data from sources like oracle, flat files and transforming them using business logic and loading the data to the business tables.• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Python and Scala.• Involvement in post-production support, Testing and used JUNIT for unit testing of the module.• Created detailed Power BI documentation, including data models, DAX formulas, and user guides, to support ongoing maintenance and user adoption• Creating Pipelines in ADF using Linked Services/ Datasets/ Pipeline to Extract, Transform, and load data from different sources like Azure SQL, SSIS, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.