Data Engineer
Current- Architect and optimize data pipelines for ingesting and transforming large volumes of financial data using AWS services (Glue, Redshift, S3) to handle critical data workflows and ensure data accuracy.
- Collaborate with compliance and legal teams to implement data governance and security frameworks that adhere to SEC and FINRA guidelines, using AWS KMS for encryption and IAM for strict access control policies.
- Spearhead deployment of real-time streaming solutions using Apache Kafka and Spark Streaming, allowing faster transaction processing and enabling front-office systems to handle high-throughput, low-latency data needs.
- Implement data quality validation within ETL workflows using SQL and Python, reducing data errors by 30% and improving the reliability of reports used by executive decision-makers and data analysts.
- Design and maintain data models in Snowflake and Redshift, optimized for financial reporting, customer analytics, and risk assessment; reduced query times by 25% by refining table structures and indexing.
- Lead initiatives to integrate machine learning capabilities into the data pipeline, supporting predictive analytics for customer behaviour and investment trend analysis, using AWS SageMaker for deployment.