1. Architectured dashboarding solution from inception using following cutting edge tools andtechnologies: Apache Pinot, Apache Spark(Pyspark), AWS Glue, Python, Airflow, Kafka, Java,Debezium (CDC tool), AWS S3, Hadoop as deltalake and datalake, KStream Java library.2. Developed comprehensive Batch and Realtime data pipelines for seamless data ingestion intoApache Pinot's high-performance real-time analytical database.SKILLS Revolutionized PySpark code and SQL queries, driving significant performance improvementsacross the data pipeline, and delivering remarkable reductions in dashboard response times.3. Delivered a substantial enhancement in user experience by decreasing response time from 6seconds per user to an exceptional 2 seconds for 100 concurrent users. Achieved an 80%reduction in the number of queries per report/dashboard page during this initiative.4. As a Data Engineering Lead, I led SQL query optimizations, conducted meticulous code reviews inPySpark and Java, fine-tuned data pipelines, and ensured a deep comprehension of functionalrequirements.5. I provided guidance to team members, mentoring them in task execution, facilitating taskassignments, and ensuring timely delivery of project tasks within each sprint undertaken by thedata engineering team. Collaborated with cross-functional teams to ensure seamless integrationwith other Lentra products.

1. During this period I have worked on the TIBCO Data Virtualization and TIBCO Data Science TeamStudio with various customers.2. Administered and optimized the Cloudera Data Platform Hadoop production cluster for maximumefficiency and performance.3. I have provided comprehensive support to customers, encompassing SSL, clustering, caching, data source connections (JDBC, ODBC, ADO.net), installations, integrations, SQL scripts, Python, and expertise in Spark, Hadoop, and Hive.4. I have gained hands on experience with Python, SQL, Pyspark, Hive, EMR, S3, Athena, SNS, Glue,Step Function, IAM, Cloudera CDP and CDH(Single and Multinode Installation , Hadoop, Spark.