T Ram Yadav Email & Phone Number

Frankfurt Am Main, Hessen, De

Leveraged Spark RDDs to process and analyze data by extracting it from HDFS, enabling predictive analytics capabilities.Utilized Hive Context to access data, leveraging the advanced functionality offered by SQLContext. Employed HiveQL queries for data retrieval from Hive tables, including fact and syndicate tables.Employed efficient data modeling techniques in Hive, extensively utilizing partitioning for enhanced data separation and faster processing. Followed industry best practices to optimize Hive performance.Developed custom Scala RDDs to perform complex data transformations and execute actions on RDDs within Spark scripts. Implemented caching of RDDs to improve performance and facilitate actions on each RDD.Designed and constructed Hive Fact tables to process raw data from multiple retailers. These tables were partitioned based on the Time dimension key, Retailer name, and Data supplier name. These tables served as inputs to analytics service engines.

Beaverton, Or, Us

Built scalable distributed data solutions in an EMR cluster environment with Amazon EMR 5.6.1.Utilized Kafka REST API and Sqoop to collect and load data into the Hadoop file system from both streaming and relational databases.Extracted real-time data feeds using Kafka and Spark Streaming, processed the data as RDDs and DataFrames, and stored it in Parquet format in HDFS.Utilized Spark Streaming APIs to perform necessary data transformations and actions on the data received from Kafka, persisting the results in HDFS.Developed custom Spark scripts in Scala, leveraging RDDs for efficient data transformations and actions.

Chantilly, Virginia, Us

Prepared comprehensive ETL design documents encompassing database structure, change data capture, error handling, restart and refresh strategies.Adapted advanced analytics, machine learning, and AI techniques to support predictive maintenance, anomaly detection, and other data-driven applications in the oil & gas industry.Optimized data infrastructure and data architecture for predictive maintenance, asset performance management, and predictive analytics for operational insights in the oil & gas Softwares..Implement data governance strategies and best practices to ensure compliance, security, and privacy of sensitive oil & gas data assets.Worked with diverse data feeds including JSON, CSV, XML, and DAT, implementing the Data Lake concept.• Developed Informatica design mappings incorporating various transformations.• Leveraged AWS infrastructure, including AWS EMR Distribution for Hadoop, AWS S3 for raw file storage, and AWS EC2 for Kafka.Utilized AWS Lambda for performing data validation, filtering, sorting, and other transformations triggered by data changes in a DynamoDB table. Transformed data was loaded into another data store.Programmed ETL functions for data integration between Oracle and Amazon Redshift.Maintained end-to-end ownership for analyzing data, developing frameworks, and implementing a range of customer analytics projects.Gained exposure to IRI's end-to-end analytics service engine and new big data platforms such as the Hadoop loader framework and Big Data Spark framework.Utilized Kafka producer to ingest raw data into Kafka topics and ran Spark Streaming applications to process clickstream events.Conducted data analysis and implemented predictive data modeling techniques.

Understood project requirements and prepared architecture documentation for the Big Data project.Supported MapReduce Java programs running on the cluster.Optimized Amazon Redshift clusters, Apache Hadoop clusters, data distribution, and data processing.• Developed MapReduce programs to process Avro files, performing calculations on the data and implementing map-side joins.Imported bulk data into HBase using MapReduce programs.Utilized REST API to access HBase data for analytics purposes.Designed and implemented incremental imports into Hive tables.Created Hive tables, loaded them with data, and wrote Hive queries running internally in a MapReduce manner.• Collected, aggregated, and moved data from servers to HDFS using Flume.Imported and exported data from various relational data sources (DB2, SQL Server, Teradata) to HDFS using Sqoop.Built applications using Maven and integrated with continuous integration servers like Jenkins.

Basking Ridge, Nj, Us

Involved in developing ETL processes to extract data from ERPs and Oracle databases, transform it using Informatica's SQ, Router, Expression, Aggregator, Lookup, and other transformations, and load it into the target HDFS environment and Teradata database. Also, handled metadata in MySQL for data loading into HDFS.Do Participated in architecture council for database architecture recommendationUtilized Unix Shell Scripts for adding the header to the flat file targets.Used Teradata utilities fastload, multiload, tpump to load data.Preparation of the Test Cases and involvement in Unit Testing and System Integration Testing.Deep analysis on SQL execution plan and recommend hints or restructure or introduce index or materialized view for better performance