Data Engineer
Current✪ Successfully designed and developed scalable data pipelines on Databricks, reducing data processing time by 30% through optimized Spark jobs and orchestration.✪ Implemented real-time data processing and streaming solutions using Spark Streaming and Delta Live Tables, increasing data integration efficiency by 40% and ensuring seamless real-time data availability for downstream reporting.► Design, develop, and manage scalable data pipelines within the Databricks environment, utilizing Spark jobs, tables, views, and orchestration, adhering to enterprise data model designs.► Implement real-time data processing and streaming solutions using Databricks tools such as Spark Streaming, Online Tables, and Delta Live Tables.► Ensure the high quality of code by establishing and maintaining robust data observability, metadata standards, and best practices across testing, code repositories, and naming conventions.► Work closely with data science, UI, and reporting teams to understand their data requirements and translate those into effective data models.► Share expertise with internal teams through presentations, technical talks, and training, while ensuring clear communication regarding architecture, execution plans, and releases.
