Data Engineer
Current• Manage and maintain data pipelines, ensuring seamless end-to-end operation and timely patching.– Oversee data pipelines, capturing changes with Kafka and Debezium for real-time synchronization from source systems to the Data Lake and Data Warehouse for storage, transformation, and aggregation.– Continuously monitor pipeline performance and implement optimizations to enhance efficiency and throughput across the entire data flow, while ensuring the data quality, security, and consistency.• Developed Demographic scoring model for loan applications to enhance decision-making and improve loan approvals.– Implemented the model as RESTful API for seamless integration with existing loan processing systems, enabling real-time scoring and automated decision-making.• Re-engineered a legacy data pipeline by piloting the development of an end-to-end ETL process using AWS Database Migration Service for Change Data Capture (CDC), AWS Glue for data integration, DBT for data transformation, and QuickSight for reporting and visualization.