Managing the various cloud pipeline architectures with 13 different architectures, 88+ data sources and PetaBytes of data lake.Build 2 real time data ingestion pipelines from scratch.

Dedicated System Engineer with expertise in designing, implementing, and securing IT infrastructures. Proven troubleshooter and collaborator, ensuring optimal system performance aligned with business goals.

- Worked with 40+ DAG files and utilized Apache Airflow in conjunction with AWS for ETL data pipelinemanagement. - Employed CloudFormation templates for infrastructure provisioning and configuration.- Crafted PostgreSQL queries for Redshift Spectrum, optimizing data retrieval and transformation processes.Ensured efficient query execution and 100% performance tuning to meet business requirements.- Implemented SMTP notification loggerfor managing Airflow task statuses and providing timely alerts for dataprocessing workflows which saved 30% manual effort.- Translated 10+ Hive commands into Redshift Spectrum, enabling seamless data querying across AWS servicesand enhancing compatibility with existing workflows.- Fabricated several automation scripts for AWS Managed Airflow. Collaborated with 3 cross-functional teams, and followed agile Scrum methodology for efficient project management.

During this boot camp, I have got to learn about machine learning and deep learning methodology and mathematics behind all the concepts. Deep learning involves various concepts like Neural Networks, Convolutional neural networks, Recurrent neural networks, sequence models, optimization algorithms, hyperparameter tuning, etc, I got to learn in this specialization curriculum. Scored all quizzes and completed all hands-on assignments successfully. Also, I have got a new perspective to observe and understand the human mind.