Data Engineering Manager
CurrentSupervised a large team of data engineers, project managers, analysts, web developers and testers ensuring to make the clients happy by designing, developing and providing successful deliverables without delay.A few projects from my experience at Serendio:1. An Azure based expense management solution for a telecom services provider using MySQL, Java and reporting using Microsoft Power BI. 2. A Streaming IOT solution for enriching data produced from IOT devices, by calling external third party services. In addition, the solution also provides notifications for real-time stream monitoring. The solution was developed using Kafka, Spark Streaming and ElasticSearch. 3. Data Engineering pipeline architecture using Hadoop HDFS, Sqoop, Hive, Impala, Spark, Kafka & HBase on both batch data and real-time streaming data.4. Campaign management system, which also involves real-time monitoring for ongoing campaigns and generates analytical reports using Angular, Node, MySQL. 5. ETL processes using Python & MySQL & Hadoop Ecosystem on structured and semi-structured datasets and reporting using PowerBI 6. Interactive Dashboard using Angular, NodeJS, MySQL. 7. Data processing operations using Spark APIs in Python using Spark SQL & Data Frames.
