Senior Solutions Consultant
Current- Developed a hybrid connect solution using Confluent for Kubernetes (CFK) to integrate on-premises Confluent Kafka Connect with Confluent Cloud.- Leveraged Confluent's Secret Provider for externalizing secrets, ensuring secure management of sensitive data across hybrid Kafka Connect deployments.- Utilized Single Message Transforms (SMT) to clean Change Data Capture (CDC) records from Debezium connectors.- Integrated RBAC into Confluent Control Center to enable authentication and secure access for the hybrid connect project.- Implemented Cluster Linking via CFK to mirror on-premises topics on Confluent Cloud, enabling efficient data replication and failover.- Successfully migrated from on-premises Confluent Platform to Confluent Cloud, ensuring seamless integration and enhanced cloud capabilities.- Created and conducted detailed workshops on Apache Flink, covering foundational concepts as well as advanced topics like stateful stream processing, windowing, and event time processing. These workshops were designed to empower teams with the skills needed to leverage Flink for real-time data processing.- Extensive Flink SQL Query Development: Wrote complex Flink SQL queries to handle various use cases, including real-time analytics, data enrichment- Designed and implemented stream processing pipelines that integrated Apache Flink with Kafka. This included setting up Flink jobs that consumed Kafka topics, applied real-time transformations, and produced results back to Kafka or other sinks.- Developed custom Flink jobs tailored to meet specific business needs, such as real-time fraud detection, monitoring, and predictive analytics. These jobs utilized advanced Flink features like keyed state, timers, and custom operators.- Managed and deployed Flink clusters on Kubernetes, ensuring high availability, scalability, and efficient resource management.