Arun M Email & Phone Number

United States

As a Senior Data Engineer I worked with Data Scientists using Amazon SageMaker to analyze data and prototype models. I managed data cataloging and metadata with AWS Glue, and used AWS Redshift for data warehousing and complex SQL queries. I created interactive dashboards with Amazon QuickSight and handled real-time data streaming with AWS Kinesis. I also.

As a Senior Data Engineer I developed and executed system test strategies for data transfer from our internal warehouse to an external entity, automated regular MySQL to Azure SQL Data Warehouse transfers, and led the integration of 142 million records into Azure SQL DW. I converted a local app to Azure-based, redesigned architecture for seamless data.

As a Data Engineer, I designed and implemented ETL pipelines to centralize sales data using Python and PostgreSQL, enhancing accuracy and efficiency. Performance tuning on MySQL improved query response times by 30% and reduced resource consumption. Additionally, I created a SQL Server data warehouse for reliable data retrieval and wrote complex SQL queries.

As a Big Data Developer, I installed and configured Hadoop components like MapReduce, HDFS, and Hive, and developed Java MapReduce jobs for data preprocessing. Using Sqoop, I imported/exported data between HDFS/Hive and other systems like Teradata. I managed cluster maintenance, ETL processes, and data analysis tasks, including Hive queries and PIG scripts.