Ajay Kumar P Email & Phone Number

New Brunswick, New Jersey, United States

• Orchestrated the creation of dynamic data processing workflows using Azure Databricks and the distributed processing capabilities of Spark, to evaluate and comprehend client behavior in real-time.• Collaborated on ETL tasks with a focus on maintaining data integrity and verifying the stability of real-time customer behavior analysis pipelines.• Worked on migration of data from On - prem SQL server to Cloud databases (Azure Synapse Analytics (DW) & Azure SQL DB). • Worked on Snowflake Schema, Data Modeling, Source to Target Mappings, Interface Matrix, and Design elements, while also designing and modifying Snowflake tables, views, and schemas to optimize retrieval and storage efficiency, ensuring a robust foundation for real-time analytics and reporting on client activity.• Performed data quality issue analysis using Snow SQL by building analytical warehouses on Snowflake.• Incorporating approaches such as Slowly Changing Dimension (SCD) and Change Data Capture (CDC) into microservices to preserve data integrity and effectively capture small changes, these pipelines smoothly interface with a range of sources, including SQL databases, CSV files, and REST APIs.• Integrated big data processing and analytics capabilities with Azure Synapse Analytics, allowing for effortless exploration and generation of real-time insights from customer behavior data.• Utilized Snowpipe to automatically ingest and process streaming data from Kafka into Snowflake, enabling real-time analysis of high-volume streaming data for immediate insights into customer behavior.• Configured Snowpipe to load data from Azure Data Lake Storage (ADLS GEN2) into Snowflake, providing a seamless integration between the data lake and the data warehouse for efficient data processing and analysis.• To meet specific business requirements wrote UDF's in Scala and PySpark. Analyzed large data sets using Hive queries for Structure.

Memphis, Tn

• Implemented strategic data processing workflows in Azure Databricks, harnessing Spark's capabilities to execute large-scale data transformations, contributing significantly to operational efficiency in managing package tracking, shipment details, and supply chain data.• SCD and CDC techniques were integrated into these workflows to manage historical and incremental changes in customer data effectively. • Implemented scalable and optimized Snowflake and Azure Synapse schemas, tables, and views, catering to complex reporting requirements and analytics queries, ultimately improving decision-making processes across the logistics and transportation landscape.• Designed data ingestion pipelines utilizing Azure Event Hubs and Azure Functions, facilitating real-time data streaming into Azure Synapse for timely insights into package tracking and shipment information.• Used Azure Data Lake Storage for efficient storage of raw and processed data, applying data partitioning and retention strategies to enhance data management and support the company's daily operations.• Leveraged Azure Blob Storage for streamlined storage and retrieval of data files, implementing compression and encryption techniques to optimize costs and enhance data security.• Integrated Azure Data Factory with Azure Logic Apps to orchestrate complex data workflows, triggering actions based on specific events and significantly improving customer service through enhanced package tracking and communication.• Implemented data replication and synchronization strategies between Azure Synapse, Snowflake, and other data platforms, utilizing Azure Data Factory and Change Data Capture (CDC) techniques to maintain data integrity.• Leveraged capabilities of Snow pipe to process semi-structured data formats such as JSON and Parquet, enabling efficient storage and analysis of complex data structures in Snowflake.

Sunnyvale, Ca

• Developed custom transformations and UDFs (User Defined Functions) in Scala and PySpark within Azure Databricks notebooks to address specific business logic requirements, enhancing data processing capabilities.• Leveraged big data ecosystems, including Azure HDInsight which includes Hadoop, Spark, and other big data technologies for loading, and transforming diverse sets of structured, semi-structured, and unstructured data.• Integrated Azure Cosmos DB with Hive within the Analytics Zone, optimizing data storage and retrieval processes.• Applied Hive queries and Spark SQL on Azure HDInsight to meet specific business requirements, employing MapReduce-like functionalities for data analysis and processing.• Implemented Azure DevOps Pipelines for automation in deployments, leading to expedited and more streamlined build and release processes.• Designed and implemented data partitioning strategies in Azure Cosmos DB to optimize query performance and reduce latency in data retrieval operations.• Developed custom transformations and UDFs (User Defined Functions) in Scala and PySpark within Azure Databricks notebooks to address specific business logic requirements, enhancing data processing capabilities.• Executed successful migration of data from RDBMS (Oracle) to Azure SQL Database utilizing Sqoop, bolstering data management and processing capabilities.• Conducted performance tuning and optimization tasks on Hive queries and Spark jobs to improve processing efficiency and reduce execution times. And Implement data modeling solutions using tools like Azure Data Lake Storage and Azure SQL Data Warehouse, ensuring scalability and performance of analytical workloads.• Utilized Azure DevOps Boards for issue management and project workflow, enhancing overall project organization and efficiency.