In my role as a Data Engineer at Amazon in New York, I worked closely with cross-functional teams to design and implement scalable data solutions that drove business impact. My key achievements include migrating on-prem ETL workloads to AWS, optimizing EMR clusters for performance, and enabling data-driven decision-making through intuitive dashboards and fine-grained analytics.

At Shutterfly, I played a key role in designing and implementing data pipelines using AWS services like DMS, S3, Glue, Lambda, and Redshift. These pipelines processed data from multiple sources, resulting in a 30% increase in data accuracy.I identified transactional systems (Systems of Record) and implemented data lineage and transformations for KPI reporting, ensuring data integrity and supporting informed decision-making. Additionally, I developed and maintained reporting dashboards in Looker, enabling stakeholders to track key performance indicators effectively. By optimizing Looker connections to Redshift, I reduced report refresh times to 1-2 minutes, significantly improving reporting efficiency.My work involved conducting comprehensive data analysis—extracting, cleansing, and manipulating data from diverse sources—to support business decision-making processes and drive operational efficiency.

At LTI, I gained extensive experience in big data processing and analytics, working with tools like Spark, Hive, and Hadoop. I implemented business transformations and anomaly detection for credit card transactions using Spark SQL and Hive, playing a critical role in fraud detection and prevention.I designed and developed ETL workflows using Oozie, automating data pipelines and reducing manual interventions, which significantly enhanced productivity and reliability. I also optimized Hive table performance by implementing partitions and bucketing, reducing query response times and improving system efficiency.Additionally, I built a robust data lake on Cloudera, transforming raw datasets into actionable formats, and automated batch processes using Unix scripting to ensure consistent and accurate data processing. My work consistently aligned technology solutions with client requirements, adding significant value to their operations.