Sagadevan K Email & Phone Number

Data Engineer with 3 years experience in AWS related data services. Good capability to understand and analyze the data, requirements and consumption patterns to develop an end-to-end solution, starting from ingestion and transformation to creating dashboards. Strong desire to move forward, face new challenges and expand my skill set.

• 

  Data Engineer

  Quantiphi

  Jul 2021 - Present - Mumbai, Maharashtra, India

  Iceberg Medallion ArchitectureBuilt modular and idempotent Glue jobs to load data from multiple APIs to AWS S3 (Bronze layer), with auditing in DynamoDB. Developed Spark Glue jobs to load deduplicated and transformed data from Bronze layer to Iceberg tables in the Silver layer. Orchestrated ETL workflows using Mage AI, ensuring process sequencing and logging in DynamoDB.ETL workflow for Data IntegrationCreated ETL pipelines for loading S3 data to Aurora Postgres with cleaning, transformations, and schema evolution. Wrote unit and E2E tests for Glue jobs, maintaining 80%+ test coverage. Automated workflows using Step Functions to refresh materialized views and QuickSight SPICE datasets.Redshift Data WarehousingDesigned Glue jobs for loading data from S3 files (DMS loaded) to Redshift staging and DWH layers, with audit logging, unit and E2E tests. Integrated Google Analytics data into Redshift using Analytics Data API and Glue jobs.On Premises to AWS migration (Data Lake + Warehouse)Managed ETL processes from multiple sources like SQL Server, APIs, and SFTP to Redshift via S3 (parquet format) using Glue jobs. Recreated SQL Server stored procedures in Redshift to maintain business logic.

• 

  Thadomal Shahani Engineering College

  Information Technology