Lead Data Engineer
Current- Modern Data Stack Development: Built and maintained a modern data stack using Fivetran, Airflow (MWAA), Snowflake, DBT, and PowerBI. Implemented configuration-driven data ingestion, data quality, error logging, and.
- Data Ingestion and Pipeline Management: Designed, deployed, and maintained data ingestion frameworks for both streaming and batch data using Fivetran, AWS Lambda, DynamoDB, S3, and AWS Glue. Orchestrated data pipelines.
- Custom Data Processing: Developed custom Python scripts to parse CSV and Excel data before ingestion into Snowflake, ensuring data integrity and quality.
- Snowflake Administration and Optimization: Managed Snowflake as an admin, creating and granting roles using RBAC. Optimized cost and performance by creating dashboards to monitor costly warehouses, long-running.
- Hackathons and Innovation: Participated in hackathons involving Amazon Bedrock/Chat-GPT for image recognition and web scraping data analysis, demonstrating adaptability and innovative problem-solving skills.