Experienced Data Engineer with over 3.5 years of expertise in Data warehousing and Pyspark. Ability to design, develop and implement data pipelines, ETL processes and data warehousing solutions using AWS Glue, AWS Lambda, Eventbridge, PySpark, S3, Opensearch, DataStage, Oracle SQL, Hive, Hadoop and ELK Stack.
-
Data EngineerIbm Oct 2024 - PresentPune, Maharashtra, India -
Data EngineerCognizant Aug 2021 - Oct 2024Pune, Maharashtra, India- Orchestrated on-premise batch data pipelines using Oracle, Elasticsearch (ELK), and DataStage, processing 2 billion data items daily and managing over 50 TB of data.- Migrated same batch data pipeline to AWS Cloud, utilizing Lambda functions for data integration into S3, processing 30 TB of data daily.- Developed AWS Glue jobs using PySpark to transform data, handling up to 1 billion data items daily.- Implemented Hadoop and Hive for efficient on-premise data storage and retrieval, then transitioned to cloud-based solutions.- Coordinated data profiling and key assignment using Lambda and EventBridge triggers for data moving from S3 to Amazon RDS.- Managed transformed data across 100+ OpenSearch indices to ensure efficient data retrieval. -
InternCognizant Apr 2021 - Aug 2021Pune, Maharashtra, India• Developed a MySQL-based software to maintain real-time movie records in stores, resulting in 50% hike in data accuracy.• Facilitated data flow by developing a high-level equivalent mapping in Informatica Powercenter, contributing to improved Data Warehousing operations.
Varad Lad Education Details
Frequently Asked Questions about Varad Lad
What company does Varad Lad work for?
Varad Lad works for Ibm
What is Varad Lad's role at the current company?
Varad Lad's current role is Data Engineer at IBM.
What schools did Varad Lad attend?
Varad Lad attended M.g.m's Jawaharlal Nehru College Of Engineering.
Who are Varad Lad's colleagues?
Varad Lad's colleagues are Wei Xu, Sudhir Singh, Veranjaneyulu Kuruva, Prabakaran Murali, Stacy Quesada, Heather Lorraine, Brendan Coghlan.
Not the Varad Lad you were looking for?
Free Chrome Extension
Find emails, phones & company data instantly
Aero Online
Your AI prospecting assistant
Select data to include:
0 records × $0.02 per record
Download 750 million emails and 100 million phone numbers
Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.
Start your free trial