Rohith N Email & Phone Number

United States

Spearheading enterprise-wide data engineering projects, I designed scalable ETL/ELT pipelines utilizing distributed processing frameworks and modular workflows, enabling seamless integration of diverse datasets. By optimizing complex transformations with tools like Apache Spark and Delta Lake, I reduced query execution times by 50%, enhancing overall system efficiency.Implemented robust real-time data ingestion frameworks leveraging event-driven architectures, achieving significant latency reductions. Designed dimensional models with star and snowflake schemas, supporting analytics initiatives with efficient query patterns and high performance.Led the adoption of advanced data processing frameworks, integrating validation and monitoring tools to improve data reliability and governance. My contributions to lineage tracking and metadata cataloging ensured traceability, improving data accessibility and compliance. Additionally, I implemented schema evolution and ACID properties in data lakes to enhance data integrity across the enterprise, achieving a 20% boost in data pipeline efficiency.

Engineered robust data pipelines and optimized workflows using distributed computing technologies, enabling scalable and reliable processing of large datasets. Developed real-time data streaming solutions with tools like Kafka and Spark Streaming, reducing processing latency by 40%.Designed batch and streaming ETL processes, integrating advanced transformation logic and automating end-to-end data flows. Leveraged DBT for modular and version-controlled SQL transformations, enabling scalable and efficient analytics frameworks. A significant achievement includes implementing a machine learning pipeline for customer segmentation, which improved campaign efficiency by 20%.I optimized relational and columnar data storage by designing star schemas and implementing materialized views, achieving a 30% reduction in query runtime. By integrating observability and governance tools, I improved data reliability, compliance, and operational transparency across workflows. My focus on CI/CD practices for data pipelines reduced development lifecycles and improved system scalability.

Designed and implemented ETL pipelines using distributed data processing tools, reducing batch processing times by 35%. Optimized Apache Spark and Hive workflows for scalable data processing, handling structured and semi-structured data seamlessly.Built hybrid database architectures to integrate OLTP and OLAP systems, ensuring efficient data retrieval and storage. Enhanced real-time data ingestion using message brokers and event-driven architectures, achieving a 25% increase in analytics timeliness.Developed slowly changing dimensions (SCDs) for historical tracking in dimensional models, improving analytics accuracy. Orchestrated workflows using tools like Apache Airflow and Oozie, ensuring robust pipeline automation and error handling.Integrated columnar storage formats like Parquet and ORC to enhance compression and query performance. My efforts in data governance and lineage tracking improved compliance, boosted data quality, and increased system reliability.

Delivered end-to-end data engineering solutions, building ETL pipelines that reduced batch processing times by 40% using tools like Hive, Pig, and Talend. Developed scalable workflows for ingesting and transforming large datasets, leveraging distributed storage and processing systems effectively.Optimized MapReduce jobs and Spark transformations to process structured and unstructured data, achieving a 30% improvement in performance metrics. Designed robust Hive external tables and partitioning strategies to enable efficient query execution and reporting.Implemented schema evolution and data governance practices, enhancing data accessibility and trust across the organization. Leveraged advanced compression algorithms and columnar formats like Avro and Parquet to reduce storage costs and accelerate processing.Created custom Hive UDFs and Spark UDFs for complex transformations, enabling data standardization and enrichment. By integrating real-time dashboards and interactive visualizations, I empowered stakeholders with actionable insights, driving data-driven decision-making across teams.