Prakash A Email and Phone Number

• Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks.• Integrate Azure storage with Azure Databricks, Build ELT pipeline using Databricks Jobs and Workflows.• Orchestrate Azure databricks applications using ADF pipelines, • Build Data pipelines using ADF pipelines and Databricks.• Developed PySpark applications and deployed in Databricks cluster. • Developed high-quality Python/PySpark code and implemented software development best practices to maintain high standards.• Created Azure Synapse Analytics Workspace, used Azure Synapse Studio, Azure Synapse Serverless SQL. • Integrating Azure Synapse Workspace and Data in ADLS Files, Creating External Tables on ADLS Files using Azure Synapse Workspace, Build ADF data flow using Azure SQL.• Extensively worked on Azure Data Lake Analytics with the help of Azure Data bricks to implement SCD-1, SCD-2 approaches.• Well versed with REST APIs and CRUD operations to access and transform analytic data. • Importing and exporting data into HDFS and hive using Sqoop and Kafka with batch and streaming. • Responsible for coding MapReduce programs, writing Hive queries, and testing and debugging these MapReduce programs.• Involved in complete Big Data flow of the application data ingestion from upstream to HDFS, processing the data in HDFS and analyzing the data using several tools. • Used Azure Data Factory, SQL API, and MongoDB API to merge data from MongoDB, MS SQL, and cloud services like Blob storage, Azure SQL DB, and Cosmos DB.• Created complex stored procedures to perform index maintenance and data profiling for loading data marts and generating datasets for reports.• Utilized Azure Data Factory to perform Azure-SSIS integration and deployed SSIS packages to cloud.• Handled structured and unstructured datasets.• Experience in integrating Teradata with Azure services, including Azure Data Factory for data migration and ETL processes.