Pradeep Pujari Email & Phone Number

1. Building Scalable ETL SolutionsDeveloped Python Flask REST APIs for ingesting metadata from diverse sources, including ERP systems, CRM systems, RDBMS, and file formats.Built PySpark-based ETL pipelines to process and ingest actual data into Hive warehouses, meeting precise business requirements.Designed AWS Glue jobs to ingest data into S3 and leveraged Athena for querying, ensuring seamless cloud integration.2. Data Management and Query OptimizationCentralized metadata management in Apache Atlas and synchronized it with Hive warehouse for on-premises systems.Created and performance-tuned complex SQL queries, supporting both ad hoc requests and dashboard development using Impala and Athena.3. Innovative Problem-Solving and CollaborationContributed to product strategy discussions, identifying challenges and proposing optimal solutions to enhance outcomes.Suggested features to frontend teams, ensuring user-friendly and efficient product designs.4. Technical Expertise in Metadata IngestionUtilized Apache Atlas REST APIs for metadata ingestion, supporting end-to-end data governance.Employed prompt engineering best practices to transform complex file formats into JSON structures, enabling meaningful metadata ingestion.5.Open-Source Integration and ManagementIntegrated, tested, and obtained stakeholder approval for open-source technologies such as Apache Atlas, Apache Ranger, and LDAP on Docker.Designed and maintained Dockerized environments using Docker Compose and CI pipelines to manage container registries effectively.6. Leadership and TeamworkCollaborated with cross-functional teams and stakeholders, aligning technical solutions with business goals.Demonstrated adaptability, problem-solving, and leadership skills while driving initiatives to simplify complexities and deliver impactful, data-driven products.

Pune, Maharashtra, India

ETL Development & Big Data Engineering:Developed scalable ETL pipelines using Hadoop, Spark, and AWS services (EMR, S3, Glue, Athena).Designed workflows to process 25TB of daily data, enabling efficient analytics and business intelligence.Automated and scheduled ETL processes to ensure timely delivery of data pipelines.Data Integration & Optimization:Integrated diverse structured and unstructured data sources for seamless workflows.Migrated workflows to AWS, leveraging Glue, S3, Athena, and EMR clusters to enhance scalability and performance.Optimized data transformations in Spark, Hive, and AWS, reducing runtime by 35%.Cloud Migration & Performance Tuning:Migrated 10TB of data from relational databases to Hadoop using Scala Spark, SparkSQL, and Hive, improving data accessibility. Streamlined cloud-based workflows with AWS services, achieving a 30% cost reduction in resource usage. Tuned complex SQL queries for dashboards and ad-hoc analytics.Cross-Team Collaboration & Deployment:Worked closely with stakeholders to integrate Apache Atlas, Apache Ranger, and LDAP for metadata and authorization management.Built Docker images for ETL pipelines and managed CI/CD pipelines for seamless deployments.Collaborated with frontend teams and delivered performance-tuned data solutions.Professional AccomplishmentsCost Optimization: Reduced AWS resource usage and S3 storage costs by 30%.Data Accuracy: Ensured 99.9% data accuracy with validation and monitoring procedures.Performance Improvement: Enhanced ETL runtime by 35% through optimized querying and transformations.Data Processing: Processed and transformed 25TB of data daily, boosting analytics team productivity.Cloud Migration: Migrated 10TB of data to Hadoop, improving speed and accessibility.

Bachelor Of Engineering - Be

Higher Secondary Certificate, Computer Science

Senior Secondary Certificate, 83.45