Mathavaraj V S Email & Phone Number

Erode, Tn, In

Kochi, Kerala, India

As a Senior Data Engineer, lead a team focused on optimizing data solutions using the latest technologies. Drive continuousimprovement by applying advanced optimization techniques and enhancing data processing pipelines. Collaborate with cross-functional teams to align technical strategies with business goals while ensuring scalable, efficient, and cost-effective datasolutions. Mentor team members, foster technical excellence, and promote a culture of innovation and continuous learning.- Leveraging advanced technologies such as Apache Pinot to modernize data streaming pipelines and boost performance.

Kochi, Kerala, India

- Conducted sales analysis for an Australian bakery cafe using AWS QuickSight, creating data pipelines from MySQL topower interactive dashboards.- Led the development of real-time and batch data processing pipelines for a leading Kerala media team, utilizing Sparkand Flink to process data from Kinesis and Kafka streams.- Implemented Snowflake and Elastic Search to efficiently transfer and enrich data, boosting dashboard performance.- Managed and scheduled projects with Airflow architecture, later migrating to a container-based setup for enhancedscalability and flexibility.- Utilized AWS Textract to extract data from soft copies of receipts, invoices, and documents.- Acquired hands-on experience with modern data services or technologies including Starrocks, Kafka, Snowpipestreaming, Apache Pinot, and StarTree.

Kochi, Kerala, India

Worked closely with clients and product managers to define requirements, and collaborated with development teams to designand implement optimal solutions.- Efficiently resolved critical issues, ensuring minimal service disruption and maintaining service quality.- Established a centralized error notification system using AWS SNS to deliver critical alerts to Slack, PagerDuty, andOpsgenie, enhancing incident management for microservices.- Leveraged Airflow to build robust workflows for EMR cluster provisioning and Spark job scheduling, including retrylogic for enhanced reliability. Implemented a comprehensive monitoring system using AWS SNS, Lambda, andCloudWatch Insights to track Airflow pipeline performance.- Developed a generic REST API using Spring Security to create and validate JWT tokens, strengthening authenticationmechanisms. Demonstrated proficiency in writing unit tests with JUnit and Mockito, adhering to clean code principles inSpring Boot development.- Developed region-specific microservices using Spring Boot to handle complex CRUD operations, implementing astandardized error structure for efficient troubleshooting. Containerized these services and Python REST APIs usingDocker, and automated deployment with GitLab Continuous Integration and Continuous Delivery (or ContinuousDeployment) CI/CD pipelines.- Expanded technical expertise through hands-on experience with Linux, Docker, and Kubernetes.

Kochi, Kerala, India

Contributed significantly to the development and optimization of Hadoop data pipelines, collaborating with pipelinedevelopment teams.- Built Airflow pipelines to orchestrate and monitor cluster operations and job execution. Worked as a Java/Big Data- Developer to implement data pipelines using HDFS, Hive, Spark, Core Java, and SQL within an Agile methodology.- Established CI/CD pipelines with Jenkins to streamline the software delivery process.- Enhanced technical skills by completing hands-on training in select AWS services: EC2, EMR, and S3.- Key contributor to the implementation of SonarQube and Checkmarx for static code analysis of Java applications,integrated into the pipeline.