Nilay Kumar Email & Phone Number

Irvine, Ca, Us

• Worked on Azure cloud platform (HDInsight, DataBricks, DataLake, Blob Storage, Data Factory, SQL DB, SQL DWH and Data Storage Explorer).• Worked on AZURE services to smoothly manage applications in the cloud.• Involved in building and creating HDInsight cluster and Storage Account with End-to-End environment.• Created Pipelines in Azure Data Factory using Linked Services/Datasets/Pipeline/ to Extract, Transform, and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool & backwards.• Developed Json Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the Cosmos Activity to load data from on prem to AZURE cloud storage and databases.• Worked on ingression source files (data, csv formats) from on-premises SQL server to Azure Data Lake Store (ADLS).• Worked on process the source files in Azure Data Lake Analytics (ADLA) using U-SQL, and generate different types of output files (Parquet, csv, data etc.)

South Jordan, Ut, Us

• Experienced in using distributed computing architectures like AWS (EC2, Redshift, and EMR, Elastic search), Hadoop, Spark, Python, and effective use of MapReduce, SQL, and Cassandra to solve big data type problems.• Worked with Spark and improved the performance and optimized the existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, RDD's, Spark YARN.• Developed and Implemented various spark jobs using AWS EMR to perform bigdata operations in AWS.• Installed the application on AWS EC2 instances and configured the storage on S3 buckets.• Utilized Spark’s in memory capabilities to handle large datasets stored on S3 Data lake.• Worked on data ingestion by going through cleansing and transformations and leveraging AWS Lambda, AWS Glue and Step Functions and loaded into S3 buckets.• Created workflows using Airflow to automate the process of extracting weblogs into S3 Data lake.• Developed and executed a migration strategy to move Data Warehouse from an Oracle platform to AWS Redshift.

Mclean, Va, Us

• Built and supported several AWS, multi-server environment using Amazon EC2, EMR, EBS, and Redshift and deployed the Big Data Hadoop application on the AWS cloud.• Involved in migrating large amounts of data from on-prem Cloudera cluster to EC2 instances deployed on Elastic MapReduce (EMR) cluster.• Implemented solutions for ingesting data from various sources and processing the Data-at-Rest utilizing Big Data technologies using Hadoop, MapReduce, Hive, HBase, and Cloud Architecture.• Designed and developed end to end ETL processing from Oracle to AWS using Amazon S3, EMR, and Spark.

Los Gatos, Ca, Us

• Worked with Hortonworks distribution. Installed, configured, and maintained a Hadoop cluster based on the business requirements and optimized the query performance using Hive• Worked on bigdata components like HDFS, MapReduce, YARN, Hive, HBase, Sqoop, Pig, and Nifi.• Used Sqoop to import and export data from HDFS to RDBMS and visualization to generate reports.• Worked on creating Hive External tables and loaded the data into tables and query data using HQL.• Worked on importing metadata into Hive using Sqoop and migrated existing tables and applications to Hive.• Worked on importing data from data sources, performed transformations using Hive and loaded data into HDFS.• Created different UDF’s and UDAF’s to analyze partitioned, bucketed data and compute various metrics for reporting on dashboard and stored them in different summary tables.• Successfully loaded files to Hive and HDFS from MongoDB, Cassandra, HBase and MySQL.