Technical Lead Java Developer
CurrentDesigned and implemented stream processing solutions using Apache Flink for real-time data analytics and ETL pipelines.Designed and implemented an end-to-end migration from Druid to Cassandra using Java and Flink.Collaborated with the operations team to deploy and monitor the Cassandra cluster, adjusting based on real-world usage patterns.Utilized Docker to containerize applications and streamline deployment on OpenShift.Contributed to the development of a real-time dashboard (RTT UI) for visualizing data and alarm statuses.Conducted local testing of Flink jobs prior to deployment, ensuring application reliability and performance.Developed and deployed Flink jobs on OpenShift, optimizing real-time data processing for large streaming datasets with minimal latency using Grafana for monitoring.Configured Flink's checkpointing and state backend to guarantee data consistency and fault tolerance during job failures or restarts.Tuned Flink job configurations such as parallelism and task slots to optimize performance and resource utilization in large-scale deployments.Built real-time data enrichment pipelines by joining streaming datasets with external reference data.Monitored and managed Flink clusters on OpenShift, ensuring resource allocation and high availability.Integrated Flink pipelines with Kafka for efficient message ingestion and processing in real-time.Developed custom Flink operators and user-defined functions to meet specific business logic requirements.Set up job recovery strategies and used monitoring tools like Prometheus and Grafana to track job health and metrics.Followed best practices for code quality, including unit testing, code reviews, and CI/CD pipelines, to maintain a robust development process.Worked in an OpenShift environment to deploy and manage Flink applications, ensuring seamless integration with existing infrastructure.