Shravan Kumar N Email & Phone Number

Atlanta, Georgia, United States

● Expertly executed data transformation in Azure Data Lake Storage Gen-2 with medallion architecture, revolutionizing workflows, elevating data quality, and driving efficient decision-making in the Azure cloud environment.● Orchestrated seamless integration between on-premises and cloud data platforms leveraging Azure Data Factory, enacting complex transformations, and optimizing data loads into Snowflake.● Designed and implemented high-performance ETL transformations and rigorous… Show more ● Expertly executed data transformation in Azure Data Lake Storage Gen-2 with medallion architecture, revolutionizing workflows, elevating data quality, and driving efficient decision-making in the Azure cloud environment.● Orchestrated seamless integration between on-premises and cloud data platforms leveraging Azure Data Factory, enacting complex transformations, and optimizing data loads into Snowflake.● Designed and implemented high-performance ETL transformations and rigorous validation processes using Spark-SQL and Data Frames within the Azure Databricks and Azure Data Factory environments.● Exceptionally collaborative team player, proactively partnering with Azure Logic Apps administrators and DevOps engineers for seamless monitoring and swift resolution of automation and data pipeline issues.● Proficient in writing and maintaining YAML configuration files for data pipeline orchestration.● Optimized Azure Functions to efficiently extract, transform, and load data from multiple sources, improving ETL performance by 60%.● Engineered scalable data pipelines leveraging Airflow, Kafka, Spark, and Snowflake optimized for performance, reliability, and real-time analytics. Show less

● Led and executed comprehensive data migration strategies, efficiently transferring mission-critical databases to Azure Data Lake, improving data accessibility and scalability.● Enhanced Spark performance by optimizing data processing algorithms, and leveraging techniques such as partitioning, caching, and broadcast variables.● Strategically orchestrated the seamless data ingestion of an extensive 3 billion records into a suite of Azure Services, leveraging Azure Databricks to achieve… Show more ● Led and executed comprehensive data migration strategies, efficiently transferring mission-critical databases to Azure Data Lake, improving data accessibility and scalability.● Enhanced Spark performance by optimizing data processing algorithms, and leveraging techniques such as partitioning, caching, and broadcast variables.● Strategically orchestrated the seamless data ingestion of an extensive 3 billion records into a suite of Azure Services, leveraging Azure Databricks to achieve a notable 40% enhancement in data processing speed.● Pioneered migrating on-premises Oracle ETL processes to Azure Synapse Analytics, substantially improving data processing efficiency and cost-effectiveness.● Directed the strategic transference of SQL databases to Azure Data Lake and SQL Data Warehouse, employing Azure Data Factory to ensure optimal data transfer and storage solutions.● Conducted ETL operations with Azure Databricks and led the migration from on-premises Oracle ETL to Azure Synapse Analytics, optimizing system performance and data management efficiency while achieving remarkable scalability and cost reduction.● Engineered and implemented high-performance ETL pipelines within Azure Databricks, processing over 3 terabytes of data daily with 99.99% data integrity● Effectively drove the utilization of Microsoft Azure services, including Azure HDInsight Clusters, Azure Data Factory, Azure Blob Storage, Azure Data Bricks, Azure Function Apps, Azure Security, and Azure Logic Apps, to advance project initiatives and maximize cloud capabilities. Show less

● Strategically architected and deployed an enterprise-grade Data Lake, facilitating advanced analytics, processing, storage, and reporting on high-velocity, large-scale data sets.● Managed and optimized data storage and loading procedures for Azure BLOB and Azure Data Lake storage systems, ensuring seamless integration with Azure SQL Synapse Analytics.● Optimized data processing efficiency using Spark SQL for Scala and Python, enabling seamless RDD to Data Frame conversions… Show more ● Strategically architected and deployed an enterprise-grade Data Lake, facilitating advanced analytics, processing, storage, and reporting on high-velocity, large-scale data sets.● Managed and optimized data storage and loading procedures for Azure BLOB and Azure Data Lake storage systems, ensuring seamless integration with Azure SQL Synapse Analytics.● Optimized data processing efficiency using Spark SQL for Scala and Python, enabling seamless RDD to Data Frame conversions, significantly accelerating analysis and system performance. Streamlined data manipulation, markedly enhancing processing speed and reliability. Show less

● Designed and implemented scalable AWS infrastructure utilizing services like EC2, S3, and Lambda for efficient data processing and storage.● Successfully implemented DBT modeling in a large-scale data warehouse migration project, transforming legacy ETL processes into modern, modular DBT models, resulting in improved data quality and reduced processing time by 30%.● Build Self-service data pipelines using AWS Services like SNS, StepFunction, Lambda, Glue, EMR, EC2, Athena, Sage Maker,… Show more ● Designed and implemented scalable AWS infrastructure utilizing services like EC2, S3, and Lambda for efficient data processing and storage.● Successfully implemented DBT modeling in a large-scale data warehouse migration project, transforming legacy ETL processes into modern, modular DBT models, resulting in improved data quality and reduced processing time by 30%.● Build Self-service data pipelines using AWS Services like SNS, StepFunction, Lambda, Glue, EMR, EC2, Athena, Sage Maker, Quick Sight, Redshift, etc.● successfully designed and implemented a robust data pipeline using Snowflake, enabling efficient processing and analysis of large-scale datasets for actionable insights.● Moved large amounts of data from AWS S3 buckets to AWS Redshift using Glue and EMR.● Successfully implemented Data Lakes on Amazon Web Services (AWS) and leveraged Amazon S3 and AWS Glue for seamless data ingestion, storage, and processing.● Analyzed large and critical datasets using EMR, Glue, and Spark. Show less

● Used AWS services like AWS Services like SNS, StepFunction, Lambda, Redshift, Glue, EMR, EC2, Athena, etc.● Utilized NumPy and pandas to efficiently process and analyze large datasets, enabling streamlined data engineering workflows.● Cleaned data and processed third-party spending data into maneuverable deliverables within specific formats with Excel macros and Python libraries.● Gathering business requirements by organizing and managing meetings with business stakeholders and IT… Show more ● Used AWS services like AWS Services like SNS, StepFunction, Lambda, Redshift, Glue, EMR, EC2, Athena, etc.● Utilized NumPy and pandas to efficiently process and analyze large datasets, enabling streamlined data engineering workflows.● Cleaned data and processed third-party spending data into maneuverable deliverables within specific formats with Excel macros and Python libraries.● Gathering business requirements by organizing and managing meetings with business stakeholders and IT analysts on a scheduled basis.● Analyzed the business requirements by dividing them into subject areas and understood the data flow.● Created and updated views with Python view controller and template language to create new functionalities for the website.● Built development environment using bug-tracking tools like Jira, Confluence, and version controls such as Git, and SVN.● Designed, deployed, maintained, and led the implementation of Cloud solutions and underlying technologies.● Developed a real-time data engineering project using Python and Apache Kafka to process high-volume data streams for seamless real-time analytics.● Experienced in writing Spark Applications in Python.● Implemented Spark using Python and Spark SQL for faster testing and processing of data.● Implemented Spark using Python and utilizing Data frames and SparkSQL API for faster processing of data. Show less