Siddharth Dwivedi Email & Phone Number

India

Building Modern AWS/Azure/OCI Data Platform Solutions

Gurugram, Haryana, India

Build Efficient and Extendable Easily Accessible Best practices to produce insight reports using lightning fast, in-memory data processing spark engine and Programmatic Workflow Management tool Airflow managing the complete data pipeline. Produce Parquet Data set - Easy & Faster querying With Monitoring & Alerts.Designed and built scalable, fault tolerance, and self-recoverable modern data solutions using AWS Cloud Services (S3, EMR, Redshift, Airflow) with Spark and Kafka… Show more Build Efficient and Extendable Easily Accessible Best practices to produce insight reports using lightning fast, in-memory data processing spark engine and Programmatic Workflow Management tool Airflow managing the complete data pipeline. Produce Parquet Data set - Easy & Faster querying With Monitoring & Alerts.Designed and built scalable, fault tolerance, and self-recoverable modern data solutions using AWS Cloud Services (S3, EMR, Redshift, Airflow) with Spark and Kafka Eco-systemManaged Petabyte scale of Redshift cluster by spinning up new Redshift Cluster and creating database and loading data into cluster managing backup, running maintenance operation, and managing workload managementResearched for AWS cost saving and implemented a smart price comparison system model.Getting business requirements through high-level design and Coordinated with business analysts and product team for design/specifications/ modifications related to the defect or enhancement of system.Building insight reports for External and internal clients measure their campaigns performance in various segment level.Engaged in development and deployment activities using deployment framework Circle Ci, Git and Docker.Built an automatic framework in order to consume third-party data.Worked with Data Science team closely and built data intelligence pipeline.Built Databricks solutions to business analytics team for their ad-hoc analysis.Worked on Redshift Migration from DC2 instance type to use RA3 instance type.Helped to build up last touch complex attribution data model. Show less

Gurgaon, India

Working for a real time online advertising platform (DSP) for buying ad spaces in real time through ad-exchange and placing add over there. Responsible for processing massive amount of data on EMR cluster using scalding and Apache spark frame work for generating many insight clients reports Working on actor based model for synchronizing S3 data with redshift and oracle.Build many automate applications for redshift cluster optimizing.Migrated data pipe line from Netezza to… Show more Working for a real time online advertising platform (DSP) for buying ad spaces in real time through ad-exchange and placing add over there. Responsible for processing massive amount of data on EMR cluster using scalding and Apache spark frame work for generating many insight clients reports Working on actor based model for synchronizing S3 data with redshift and oracle.Build many automate applications for redshift cluster optimizing.Migrated data pipe line from Netezza to in house Hadoop cluster and modern AWS processing systemDoing many ad-hoc R&D for report optimizations. Show less

Gurugram, Haryana, India

Noida Area, India

Worked as a dev-ops engineer in reporting team.

B.Tech, Information Technology, A

12Th, Science, A