Data Engineer Fellow
CurrentInsight is a professional training program for data engineers.Developed SafePay, scalable and extendable framework for fast and safe P2P payments. Ingested 44M Venmo transactions via Kafka, categorized in Spark Streaming and persisted in Cassandra. Ran PySpark batch job to detect anomalies. Solution coded in Python and deployed on AWS EC2 11 node cluster. Cassandra data model optimized for fast ingestion and partitioned for efficient batch processing. Monitoring dashboard built in Flask and jQuery. Project was implemented in 4 weeks start to finish.Led presentations on big data topics: Hadoop, Spark, Cassandra, Graph databases, Enterprise Architecture and System Design