Md Attaullah Email and Phone Number

• Monitored ETL processes in Azure Data Factory (ADF), managing data flows from various sources into Azure Data Cloud, handling datasets totaling up to several terabytes.• Developed and executed PySpark and SparkSQL transformations within Databricks, utilizing Delta Lake for optimized data storage and management, and applying techniques like cache, persist, and broadcast join for performance enhancement.• Oversaw Triggers/Schedules in ADF to ensure reliable and timely execution of ETL processes.• Managed Automated Job Clusters in Databricks, which are specifically used for running jobs and terminate upon completion to reduce resource usage and costs.• Managed data storage in Azure Data Lake, handling both raw and refined containers, and implemented data modeling techniques to structure data effectively.• Collaborated across 10 work streams to ensure seamless ETL operations, data processing, and data modeling.