Kumar R. Email & Phone Number

New York City Metropolitan Area

Responsibilities: • Involved in building scalable distributed data lake system for confidential real time and batch analytical needs. • Worked on data processing and transformations and actions in spark by using Python (PySpark) language. • Provisioning and configuring EC2 instances, S3 buckets, and other AWS services for data storage, processing, and analytics. • Developing and deploying serverless functions using AWS Lambda and API Gateway for real-time data… Show more Responsibilities: • Involved in building scalable distributed data lake system for confidential real time and batch analytical needs. • Worked on data processing and transformations and actions in spark by using Python (PySpark) language. • Provisioning and configuring EC2 instances, S3 buckets, and other AWS services for data storage, processing, and analytics. • Developing and deploying serverless functions using AWS Lambda and API Gateway for real-time data processing and analytics. • Building and deploying data pipelines using AWS Glue, AWS Data Pipeline, and Amazon Kinesis for batch and real-time data ingestion and processing. • Wrote and executed various MYSQL database queries from Python using Python-MySQL connector and MySQL DB package. • Development level experience in Microsoft Azure providing data movement and scheduling functionality to cloud-based technologies such as Azure Blob Storage and Azure SQL Database. • Setting up and configuring Snowflake accounts, warehouses, and databases for storing and processing data. • Developing and deploying ETL pipelines using Snowflake and tools such as Talend or Modillion for moving and transforming data • Migrating data from Oracle to Data Lake using Sqoop, Spark. • Developed Scala scripts, UDFs using both Data frames/SQL and RDD in Spark 2.4 for Data Aggregation, queries and writing data back into OLTP system through Sqoop. • Implement HIVE UDF’s for evaluation, filtering, loading, and storing of data. • Experienced in handling large datasets using partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective & efficient Joins, Transformation and other during ingestion process itself. • Worked on a POC to compare processing time for Impala with Apache Hive for batch applications to implement the former in project. • Developed logical and physical data models that capture current state/future state data elements and data flows. Show less

Portland, Oregon, United States

Responsibilities: • Involved in review of functional and non-functional requirements. • Implemented data integration solutions with traditional ETL/ELT tools and Big Data technologies. • Developed ETL mappings to Extract Data from OLTP Systems/Files applying Technical and Business Transformations loading data into Oracle Data Mart’s and Enterprise Data Warehouse Systems. • Extensive Data analysis, profiling and testing activities to deliver Quality Data Product(s)… Show more Responsibilities: • Involved in review of functional and non-functional requirements. • Implemented data integration solutions with traditional ETL/ELT tools and Big Data technologies. • Developed ETL mappings to Extract Data from OLTP Systems/Files applying Technical and Business Transformations loading data into Oracle Data Mart’s and Enterprise Data Warehouse Systems. • Extensive Data analysis, profiling and testing activities to deliver Quality Data Product(s) for Business. • Developed reusable transformations and mapplets for multiple mappings to automate and simplify development of multiple mappings. • Handled extensively the development of informatica mappings to support data extraction, transform and loading • Designed and developed a robust and efficient ETL process • Creating data lakes using AWS Lake Formation and Amazon S3 for storing and managing large amounts of structured and unstructured data. • Implementing security and access controls using AWS Identity and Access Management (IAM) and AWS Key Management Service (KMS). • Developing and deploying machine learning models using Amazon Sage Maker. Expert in using Databricks with Azure Data Factory (ADF) to compute large volumes of data. • Performed ETL operations in Azure Databricks by connecting to different relational database source systems using JDBC connectors. • Developed tools using Python, Shell scripting, XML to automate some of the menial tasks. • Developed multi-threaded standalone app in Python, PHP, C++ to view Circuit parameters and performance. • Developed scripts and batch jobs to schedule various Hadoop Programs. • Hadoop and its ecosystem (MapReduce, Hive, Sqoop. etc.) • Moved data from HDFS to RDBMS and vice versa using SQOOP. • Implemented Partitioning, Dynamic Partitions in HIVE. Show less

India

Responsibilities: • Experience in installing configuring and using Hadoop ecosystem components. • Experience in administration installing upgrading and managing CDH3 Pig Hive HBase • Importing and exporting data into HDFS and Hive using Sqoop. • Experienced in defining job flows. • Knowledge in performance troubleshooting and tuning Hadoop clusters. • Experienced in managing and reviewing Hadoop log files. • Load and transform large sets of… Show more Responsibilities: • Experience in installing configuring and using Hadoop ecosystem components. • Experience in administration installing upgrading and managing CDH3 Pig Hive HBase • Importing and exporting data into HDFS and Hive using Sqoop. • Experienced in defining job flows. • Knowledge in performance troubleshooting and tuning Hadoop clusters. • Experienced in managing and reviewing Hadoop log files. • Load and transform large sets of structured semis structured and unstructured data. • Responsible to manage data coming from different sources. • Analyzed the SQL scripts and designed it by using PySpark SQL for faster performance. • Worked on reading and writing data formats using PySpark. • Manage configuration of Web App and Deploy to AWS cloud server. • Experienced in automating, configuring and deploying instances on AWS, cloud environments and in Data centers • Implement code in Python to retrieve and manipulate data. • Supported Map Reduce Programs those are running on the cluster. • Involved in loading data from UNIX file system to HDFS. • Involved in creating Hive tables loading with data and writing hive queries which will run internally in map reduce way. • Created HBase tables to store variable data formats of PII data coming from different portfolios. • Load and transform large sets of structured semis structured and unstructured data • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team. • Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop. Show less