Data Engineer
Current• Orchestrated the deployment of Amazon EC2 instances and skillfully managed S3 storage, customizing instances for specific applications and Linux distributions.• Offered expert insights on AWS system architecture, implementing rigorous security measures and protocols to safeguard vital assets and ensure compliance.• Optimized data processing using Amazon EMR and EC2, enhancing system performance and scalability.• Designed and fine-tuned data processing workflows utilizing AWS services like Amazon EMR and Kinesis, ensuring efficient data analysis.• Leveraged AWS Glue for data cleaning and preprocessing, performed real-time analysis using Amazon Kinesis Data Analytics, and harnessed services like EMR, Redshift, DynamoDB, and Lambda for scalable data processing.• Implemented robust data governance and security protocols through AWS IAM , ensuring protection of sensitive data.• Utilized AWS Glue and Fully Managed Kafka for efficient data streaming, transformation, and preparation.• Established and managed data storage on Amazon S3 and Redshift, ensuring accessibility and organization of data.• Seamlessly integrated Snowflake into the data processing workflow, elevating data warehousing capabilities.• Evaluated on-premises data infrastructure for migration opportunities to AWS, orchestrated data pipelines via AWS Step Functions, and employed Amazon Kinesis for event-driven processing.• Proficiently utilized Apache Airflow for workflow automation, orchestrating complex data pipelines and automating tasks to boost operational efficiency.• Fine-tune Kafka configurations to optimize performance, throughput, and latency.