1) Developed and optimized high-volume data pipelines and ETL/ELT processes using tools like Spark, Airflow, and Python, achieving up to 45% reduction in data processing times and improved data quality by 35%.2)Designed scalable, cost-effective data warehousing solutions on AWS EC2 and S3, integrating diverse data sources and enhancing data flow efficiency by 30%, resulting in reduced storage costs.3)Leveraged data analytics and visualization tools such as Power BI, Tableau, and… Show more 1) Developed and optimized high-volume data pipelines and ETL/ELT processes using tools like Spark, Airflow, and Python, achieving up to 45% reduction in data processing times and improved data quality by 35%.2)Designed scalable, cost-effective data warehousing solutions on AWS EC2 and S3, integrating diverse data sources and enhancing data flow efficiency by 30%, resulting in reduced storage costs.3)Leveraged data analytics and visualization tools such as Power BI, Tableau, and Python to derive valuable insights and enhance customer retention by 30% through targeted campaigns and improved reporting. Show less

1) Extensive expertise in Azure platforms such as HDInsight, Databricks, Data Lake, and Azure Data Factory, designing automated pipelines and custom-built input adapters, optimizing data ingestion and ETL processes.2)Developed scalable ETL workflows using Spark, Sqoop, PySpark, and SQL to transfer data from on-premise systems and cloud storage to Azure Data Lakes, integrating varied data sources and automating daily incremental loads.3)Engineered automated, scalable data… Show more 1) Extensive expertise in Azure platforms such as HDInsight, Databricks, Data Lake, and Azure Data Factory, designing automated pipelines and custom-built input adapters, optimizing data ingestion and ETL processes.2)Developed scalable ETL workflows using Spark, Sqoop, PySpark, and SQL to transfer data from on-premise systems and cloud storage to Azure Data Lakes, integrating varied data sources and automating daily incremental loads.3)Engineered automated, scalable data pipelines using Azure and AWS, facilitating real-time analytics and reporting via Power BI dashboards, with a strong focus on continuous integration/deployment (CI/CD) for cloud environments. Show less