Naveen Kumar Email & Phone Number

Jersey City, Nj, Us

Jersey City, Nj, Us

Analyze, design, and build modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementations on existing business processes.Worked with Businesses Analyst, Technical Architects and Project Managers on a daily basis to get the project requirements and analyzed these requirements and then converted the business document to a technical requirement document that enlists all the technical architecture and components of requirement.Lead the team of 4 which includes both onsite and offsite resources in developing the new or maintaining existing pipelines.Created pipelines in Azure Data Factory using linked services to various source systems and then used databricks notebook for transformation and then loaded the data into azure blob storage.Created multiple realtime data ingestion scripts and batch ingestion scripts using PySpark to meet specific business requirements.Developed Databricks Asset Bundle yaml scripts for deploying pipelines in Azure Databricks.Performed the optimisation using broadcast variables and broadcast joins and used incremental loads for job processing.Developed SQL Scripts for data validation in databricks.Used PySpark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats to uncover customer usage patterns.Creating data models for bringing in the new data source for supporting various processes.Co-ordinated with various teams for on time release code into production environment.Environment: Azure Databricks, Azure Data Factory, Azure Blob Storage, Azure Logic Apps, Pyspark, SQL

Client: BCBS FLResponsibilities:Analyze, design and build modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementations on existing business processes.Data Ingestion Azure storage services and processing the data in azure databricks.Created pipelines in Azure Data Factory using linked services to databricks for extraction, transformation and loading data from various azure storage sources.Created UDFs in Scala and PySpark to meet specific business requirements. Develop JSON scripts for deploying pipelines in Azure Data Factory (ADF) that process data using SQL activities.Developed SQL Scripts for data validation in databricks.Used PySpark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats to uncover customer usage patterns.Creating data model for bringing in the new data source for supporting various process.

Schaumburg, Il, Us

Client Description:Zurich is a global insurance company which is organized into three core business segments: General Insurance, Global Life and Farmers. Zurich North American Insurance is an commercial property-casualty insurance provider. Its solutions server diverse set of industries including automotive, construction, manufacturing, technology and numerous others.Responsibilities:Migration of ETL process into Hadoop using spark and Scala.Optimizing of existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames and Pair RDD's.Used HiveQL for migrating the smaller legacy process into meld.Bringing streaming data using Apache Kafka and Spark StreamingBuilding the required data lake in Hadoop using Sqoop from various data sourceExperienced in developing custom input formats and data types to parse and process unstructured and semi structured input data and mapped them into key value pairs to implement business logic in Map-Reduce.Creating data model for bringing in the new data source for supporting various process.Bundling and compiling the code for creating jars using SBTUnit testing on data and improvement of performance, turned over to production

Us

Project 1: Cargo-jet Airways Ltd, ON, Canada - RemoteResponsibilities:Installed and configured Hadoop Mapreduce HDFS Developed multiple MapReduce jobs in java for data cleaning and preprocessing.Installed and configured Pig and also written PigLatin scripts.Involved in managing and reviewing Hadoop log files.Imported data using Sqoop to load data from MySQL to HDFS on regular basis.Developing Scripts and Batch Job to schedule various Hadoop Program.Creating Hive tables and working on them using Hive QL.Importing and exporting data into HDFS and Hive using Sqoop.Involved in creating Hive tables loading with data and writing hive queries which will run internally in map reduce way.Developed a custom FileSystem plug in for Hadoop so it can access files on Data Platform. This plugin allows MapReduce programs HBase Pig and Hive to work unmodified and access files directly.Designed and implemented Mapreduce-based large-scale parallel relation-learning systemExtracted feeds form social media sites such as Facebook Twitter using Python scripts.Setup and benchmarked HBase clusters for internal use Project 2 : Charles Schwab, TXResponsibilities:This role includes working various application team to ensure technical integrity and consistency of solutions. Creation of Queries to take in a business request for a new data extract and then code a statement that will return the desired values.Performed requirement gathering and specifications analysis with clients and implementing POCs.Processed data in HDFS by developing solutions, analyzed the data using mapreduce, Hive and produce summary results from hadoop to downstream systems.Created Flows to ingest data from various systems/sources to HDFSInvolved in creating tables and then apply HQL on these table for data validationMoved data from hive tables to mongo DB Reviewed Hadoop log files.Involved in loading and transforming large sets of data and analyze them using Hive queries and scripts.

Mechelen, Be

Description:Telenet Group is the largest provider of cable broadband services in Belgium. Its business comprises the provision of analog and digital cable television, fixed and mobile telephone services, primarily to residential customers in Flanders and Brussels. ADT is the application that was built on Customer Service Framework and it has capability to manage B2C and B2B interactions. Responsibilities: This application consolidates relevant customer information from Telenet systems, interaction data, and service requests into a composite view.They dynamically display customer data based on the customer context and current situation.Developed multiple MapReduce jobs in java for data cleaning and preprocessing involved in importing data into HDFS and Hive.Defined job flows.Involved in loading and transforming large sets of structured and unstructured data.Loaded data from UNIX system to HDFS