Software Engineer - Data
Current• Build, automate, and manage ETL processes (Airflow, Spark, Python) to create data pipelines. Perform ETL on disparate data sources – e.g., online transaction data, web service data, payment, order history, and marketing campaigns data to provide customer insights. • Automate processes that were previously manually performed using Airflow, AWS storage, database, and technologies; enhance supportability of products through CloudWatch alerts. • Oversee end-to-end experimentation. Own an entire application from development to production for successful go-live which involve ensuring co-ordination between onshore/offshore CRM, data engineering, reporting, and testing teams. • Tech lead in a new data science/ML team to productionize ML algorithms. Restructure existing data quality and profiling product framework to accommodate complex business requirements and data science use cases. • Routinely communicate with clients to update status of work, and strategically advise and manage workload expectations. • Contribute to internal open-source projects. • Establish and mentor new offshore and onshore team members.• Implement and design solutions on Adobe Experience Platform.