Cloud Data Engineer
Current• Led development of Matillion-based data pipelines, integrating AWS S3 and RDS to handle high-volume data ingestion and transformation.• Implemented Change Data Capture (CDC) using AWS tools, ensuring consistent, up-to-date data across platforms for analysis and reporting.• Built event-driven architectures using AWS Lambda, reducing latency in data processing for near real-time updates.• Designed Kimball dimensional models for reporting needs, enabling efficient data slicing and dicing for business insights.• Established thorough ELT documentation, including source-to-target maps, data dictionary, and ETL procedures for seamless knowledge sharing.• Optimized SQL code and stored procedures, improving the speed and reliability of data retrieval from AWS Postgres and Oracle Database.• Conducted root cause analysis for data discrepancies, implementing solutions to maintain data integrity across environments.• Developed REST API integrations for real-time data ingestion, efficiently handling JSON responses to streamline data flows.• Configured and monitored data pipelines using AWS CloudWatch, improving system reliability and proactively managing alerts.• Enhanced reporting capabilities by creating Power BI and OBIEE dashboards, providing interactive, user-friendly insights.• Automated infrastructure deployments with CloudFormation, ensuring rapid, consistent environment setups for ELT processes.• Collaborated with data scientists and analysts to optimize data processing workflows, improving the accuracy of analytics outputs.