- Streamlining existing pipelines to minimize operational costs.- Enhancing query optimization techniques for improved performance.- Implementing strategies to alleviate cluster workload and enhance efficiency.- Proficient in version control using Git, conducting thorough PR reviews for both new and existing Directed Acyclic Graphs (DAGs).- Crafting and scheduling DataBricks notebooks to streamline data workflows.- Leveraging Snowflake for ad hoc queries and efficient data manipulation.- Utilizing Airflow for DAG monitoring and orchestration to ensure smooth data workflows.Tools :- Pyspark(Databricks), SQL, SnowFlake, Python, GitHub, Airflow

- Collaborated closely with essential stakeholders to address their specific requirements.- Developed interactive dashboards utilizing Looker Studio (formerly Google Data Studio).- Employed uWorc for designing and managing data pipelines, ensuring seamless data orchestration.- Leveraged Hive for handling and manipulating large-scale datasets effectively.- Utilized Pyspark to conduct intricate and large-scale data manipulations efficiently.- Engineered an ML model aimed at minimizing customer churn, showcasing proficiency in machine learning within data engineering contexts.