Software Engineer Data Engineering
Current● Improved productivity by creating platforms to deliver reliable realtime front-end app data to different teams (product, data analytics, MLEs) for their different data processes.● Reduced operational errors and improved data quality by designing, developing, validating, and maintaining multiple ETL pipelines using SQL, Scala, Databricks, Airflow, Kibana, Jenkins, Datadog, and AWS.● Reduced teams’ response times by maintaining and meeting SLAs for different datasets ensuring predictable, reliable data availability.● Improved core app latency by leading a project to migrate data architecture from monolithic architecture to a more robust microservice architecture in Ruby for event logging.● Implemented image-based search and improved search recommendations latency by researching, configuring, and load-testing vector database (Milvus).● Work cross-functionally with different stakeholders (analytics, engineering, PMs, etc.) to set expectations, clarify requirements and establish timelines for instrumentation and data pipeline creation for products including SmartAI List, Consignment, etc.