As a seasoned Cloud and Data Engineer with 3 years of hands-on experience, I have developed a robust skill set in big data processing, data warehousing, and cloud resource management. My expertise lies in leveraging open-source tools like Apache Spark to process and analyze large datasets within cloud environments, as well as designing and implementing data warehouses and data lakes to support enterprise-level platforms.I have a strong focus on optimizing data pipelines, processes, and architectures to enhance operational efficiency while maintaining cost-effectiveness. My commitment to data governance and security ensures that company and customer data is always compliant and well-protected.My practical experience in managing cloud resources using Infrastructure as Code (IaC) tools, such as AWS CloudFormation, AWS CDK, and CDKTF, has allowed me to automate and streamline resource management, ensuring consistency across deployments.I have successfully developed custom monitoring solutions using Azure service APIs, which have provided deeper insights beyond standard Azure Monitor capabilities. These solutions have not only optimized job performance by identifying anomalies but also enabled real-time cost surge detection, contributing to a 5-10% monthly reduction in unnecessary expenses.In addition to managing Azure Databricks Workspaces, where I handle everything from cluster configuration to ETL scheduling, I’ve also led the transition of multiple workspaces to Databricks UC environments, enhancing data governance through the implementation of advanced tools for permissions, data lineage, and secure data sharing.Furthermore, I have secured Azure Synapse environments by implementing role-based access control and optimizing performance with serverless capabilities. My development of automated backup ETL processes and rapid workspace creation has ensured business continuity and smooth database operations.In summary, my experience spans a comprehensive range of cloud and data engineering tasks, consistently delivering solutions that enhance efficiency, security, and cost-effectiveness for the organizations I support.