Aws Data Engineer
Current• • Developed and implemented end-to-end data pipelines, enabling seamless data flow from various sources to storage and analytical systems, resulting in improvement in data availability and increase in data accuracy• Developed and managed ETL workflows using tools like AWS Glue, Apache Spark, and Apache Airflow, reducing data processing time • Designed and built data storage solutions like data lakes, data warehouses, and data marts on AWS• Designed and built real-time data streaming solutions using technologies like Kinesis and Lambda, leading to reduction in time-to-insight• Worked in data governance and data quality frameworks and implemented them in AWS environments.• Implemented error handling and retry mechanisms within Amazon Step Functions to ensure robust workflow execution, and reduce workflow failures • Created and maintained scalable and fault-tolerant data architectures in AWS using services such as S3, Redshift, and RDS.• Developed and deployed data models and schema designs for various applications and services, ensuring data accuracy and consistency.• Optimized Amazon DynamoDB tables by carefully utilizing on-demand capacity mode, and implementing data retention policies which resulted in reduction in operational costs, while maintaining optimal performance and data durability• Developed and executed data migration strategies from on-premises systems to AWS cloud-based systems, resulting in improved data accessibility and reduced infrastructure costs.• Utilized AWS Machine Learning services such as Sage Maker and Recognition to develop predictive models and image recognition solutions..