Implemented the Infrasture as a service using Terraform and Serverless framework which includes the setup of Redshift data warehouse, real-time enrichment framework, databricks workspace, and user permissions.Implemented CI/CD pipelines using GitHub Actions and GitLab pipelines for multiple projects which includes deployment of lambda code, spark-streaming jobs, DBT code, and others.Developed a bunch of batch ETL pipelines for sources like SQL databases, Stripe, Netsuite, Chargebee, Qualtrics, Zendesk, and many others using AWS services and Fivetran.Developed a set of streaming applications using Apache Spark Streaming, Apache Kafka, and Scala on databricks to implement a near real-time streaming pipeline based on S3.Developed multiple dashboards on DataDog to track the progress of streaming applications.Developed multiple dashboards on Looker like MAU, WAU, Event registration progress, and many others.Developed an end-to-end enrichment pipeline of user activity tracking events using AWS services like Lambda, Redis, SQS, and Segment.io.

Designed and implemented a travel domain data lake using the Hadoop ecosystem.Managed data analytics using Apache Hive and Apache Spark.Implemented an optimized real-time streaming platform for an OTT video domain application using Apache Kafka and Cassandra.Initiated a search engine for an OTT video domain application using Elasticsearch and Java.Developed an ETL solution using AWS Kinesis and AWS Redshift.