Senior Devops Engineer
CurrentKey Responsibilities:• Implemented Automated Backup for Databricks: Developed and deployed Lambda functions to automate daily backups of Databricks Workspaces, ensuring data integrity and recovery.• Monitored Lambda Functions: Created Lambda functions to monitor the execution status of other Lambda functions, enhancing operational efficiency and error handling.• Orchestrated Databricks Pipeline: Designed and implemented an end-to-end Databricks pipeline using AWS Step Functions, Lambda functions, Amazon EventBridge, DynamoDB, and S3 for seamless data processing and workflow automation.• Deployed Node.js Application: Successfully deployed a Node.js application using AWS Amplify and Cognito, providing robust user authentication and scalable hosting solutions.• Developed shell scripts to create and delete Amazon EKS clusters and service accounts, streamlining cluster management and reducing manual workload.• Implemented GitOps by bootstrapping GitLab with EKS clusters using FluxCD, enabling continuous delivery and automated cluster configuration management.• Utilized Helm charts to deploy and manage Elasticsearch, Kibana, and Fleet on Kubernetes nodes, ensuring scalable and efficient log management and monitoring.• Wrote CloudFormation templates to create AWS roles with multiple policies, automating the provisioning of permissions and enhancing security compliance.