Nikhil Kumar Email & Phone Number

End-to-end data pipeline development in Azure ML Workspace and Azure Data Factory (ADF)Extract, transform, and load data from diverse sources (Azure SQL, Blob storage, Azure SQL Data Warehouse, write-back tools)Spark application development using Pyspark and Spark-SQL for data extraction, transformation, and aggregationPerformance tuning of Spark applications (batch interval, parallelism, memory tuning)Azure platform expertise: Azure ML Workspace, Azure Data Factory, Azure Data Lake Storage Gen2Cluster sizing, monitoring, and troubleshooting of Spark Databricks clustersJSON scripting for pipeline deployment in Azure Data Factory (ADF)Data integration using Logic Apps, connecting Workday to various destinationsCustom alerts platform development for monitoring data workflowsData storage and retrieval in Azure Data Lake Storage Gen2, handling different file formats

Python-based Spark application development for data processing from RDBMS and streaming sourcesPerformance optimization of existing algorithms in Hadoop using Spark Context, Spark-SQL, Spark MLlib, Data Frames, Pair RDDs, and Spark YARNTuning of Spark Applications, including batch interval time, parallelism, and memory tuningUtilization of Spark Streaming APIs for real-time transformations and actions on data from Kafka, persisting it to CassandraScheduling Spark/Scala jobs using Oozie workflow in Hadoop ClusterDevelopment of Python Kafka consumer APIs for consuming data from Kafka topicsProcessing XML files and capturing UI updates using Kafka and Spark StreamingPractical implementation of cloud-specific technologies, such as IAM, Amazon EC2, ElastiCache, S3, CloudFormation, VPC, Route 53, Lambda, Glue, and EMRMigration of on-premises applications to AWS, leveraging services like EC2 and S3 for small data sets processing and storage

Analyzing large datasets to determine optimal aggregation and reporting strategiesScala-based Spark application development for data processing from RDBMS and streaming sourcesPerformance optimization of existing algorithms in Hadoop using Spark Context, Spark-SQL, Spark MLlib, Data Frames, Pair RDDs, and Spark YARNReal-time data processing using Spark Streaming and integration with Kafka, HDFS, and NoSQL databases like HBase and CassandraUtilization of Kafka for live streaming data and performing analytics, leveraging Sqoop for data transfer between relational databases and HadoopData loading from web servers and Teradata using Sqoop, Flume, and Spark Streaming APIMapReduce/Spark Python module development for machine learning, predictive analytics, and distributed random forest on AWSDesign and deployment of multi-tier applications on AWS, including services like EC2, Route53, S3, RDS, DynamoDB, SNS, SQS, IAM, with a focus on high availability, fault tolerance, and auto-scaling using AWS CloudFormationAWS Lambda development in Python for processing nested JSON files, conversions, comparisons, and sorting

• Extensive experience with Azure cloud platform, including HDInsight, Data Lake, Databricks, Blob Storage, Data Factory, Synapse, SQL, SQL DB, DWH, and Data Storage ExplorerData extraction, transformation, and loading using Azure Data Factory, T-SQL, Spark SQL, and U-SQL in Azure Data Lake AnalyticsIngesting data into Azure services (Data Lake, Storage, SQL, DW) and processing it in Azure DatabricksCreation of pipelines in Azure Data Factory (ADF) using Linked Services, Datasets, and Pipelines for data extraction, transformation, and loading from various sourcesDesigning and configuring Azure Cloud relational servers and databases, analyzing current and future business requirementsPowerShell scripting for automating Azure cloud system creation and resource managementMigration of data from on-prem SQL server to Azure cloud databases (Synapse Analytics (DW) and SQL DB)Configuration of Azure Functions with Azure Cosmos DB for data reading and writingDesigning and deploying data pipelines using Data Lake, Databricks, and Apache Airflow

Hyderabad, Telangana, India

• Analysed, designed, and developed databases using ER diagrams, normalization, and relational database concept. • Involved in design, development, and testing of the system. • Developed SQL Server stored procedures, tuned SQL queries (using indexes and execution plan). • Developed user defined functions and created views. • Created triggers to maintain the referential integrity. • Implemented exceptional handling. • Worked on client requirement and wrote complex SQL queries to generate crystal reports. • Created and automated the regular jobs. • Tuned and optimized SQL queries using execution plan and profiler. • Developed the controller component with Servlets and action classes. • Business components are developed (model components) using Enterprise Java Beans (EJB).