Senior Data Engineer
CurrentData Engineering● Technical lead and scrum master for a team of 11 engineers (3 in the US, 8 in India) for a data-driven communications system managing customer relationships for over 50 million users of printers, PCs, and mobile apps.● Developed ETL pipelines using Databricks and AWS to process data from mobile apps, PCs, and IoT devices, ensuring data is accessible via REST API for personalized app experiences and communications while maintaining privacy standards.● Reduced product onboarding to first communication delay by 92% through Spark Structured Streaming.● Achieved over 30% cost reduction in Databricks and AWS using Spark optimization, cluster configuration, and data load pattern changes.● Migrated legacy data infrastructure from AWS Data Pipelines and AWS Redshift to Databricks and Unity Catalog.● Standardized pipeline design with automated testing, Azure DevOps, CI/CD, Terraform, and Splunk dashboards.● Coordinated global delivery team, planned program increments, ran scrum ceremonies, and interfaced with stakeholders.● Collaborated with data architects to integrate data from separate divisions to launch Device as a Service business model.● Mentored and trained junior data engineers in development skills and best practices.