Arjun Kumar Email & Phone Number

Santa Clara, Ca, Us

Cupertino, California, Us

• Data Migration from On-premise Data lake to | s3 | Iceberg using Airflow DAGs | Spark | Distcp.• Data Migration from Kafka Streams to Iceberg using Flink.• Automated the Data Validation and reconsilation process.• Setting up Flink data pipeline and Data Dog Alerts and Monitoring process.• Performance tuning of flink application and reducing the Lag to meet the SLA.• Automated the process to generate the DAG for Data Copy.

Oakland, California, Us

• Creating Road Map for Data ingestion and Data Procession on Hadoop.• Designing and Building the Reporting Layer on Big Data.• Setting up Next Gen / S5 /PCR / BCABL Lab/Machines and Torrent Server.• Helped Data scientists to analyze the Cancer patient/Gene/Blood Culture using Perl and R Module.• Working closely with the Data Scientist team in Genetics Lab to analysis the Genetics data produced by iron chef and other lab machine. Helping them to identify the Cancer test result, Identify the mutant Gene, Analyze the molecular pattern and plot the graph on library_jst, alignment, polyconal and primer_dimmer.• Created the UI to Build Data Pipeline to ingest the Data from multiple source system to Hadoop.• Increase the Lab Productivity by getting the near to real time data on Test Not Done Case. • Created the Multiple Metric to measure the Lab efficiency i.e. Turnaround time and Ordered to Collected Metric.• Installed and configured Horton work Cluster (HDP 2.4/Apache 2.7) in Production/Pre-Prod and Dev environment.• Designed and Created the data flow pipeline to ingest the data from Clarity(Teradata), Pathology, Cytology and Microbiology and Genetics data to Hadoop from Teradata, Sql Server and Penguin Server using Generic Sqoop Scripts.• Develop automation tools for monitoring, management, failure detection, performance monitoring and auto remediation, Review develop, and implement strategies that preserve the data integrity, accessibility as in the current data source, interact with business process owners, data architect and other operation teams to develop and deploy solutions to meet the requirement.• Doing POC, to test the new file format, tools and frameworks

Santa Clara, Ca, Us

• Creating Ingestion -ETL Process to Ingest Data from different source system to Hive/Hbase Tables.• Creating Azkaban job flow to load daily/ Incremental Data.• Automating Sqoop Jobs to import the data and to load into hdfs.• Generating reports using Impala/hive/Python/Scala.• Ingesting Data from heterogeneous source system and creating the framework for incremental load and data validation.• Analyze current data sources, schema based on use case documentation provided, Develop program and scripts to complete data ingestion into Hadoop cluster.• Develop automation tools for monitoring, management, failure detection, performance monitoring and auto remediation, Review develop, and implement strategies that preserve the data integrity, accessibility as in the current data source, Interact with business process owners, architects and other operation teams to develop and deploy solutions to meet the requirement,.• preparation of architecture, design and operation documentation as instructed by program manager and architects.

Mclean, Va, Us

• Creating ETL Process to Move Data from Landing Phase/Key Phase/Split to Hive Tables.• Historical loads (one-time) from Teradata using an unload script built on Teradata Parallel Transporter (TPT) and Fast Load Scripts.• Creating Talend Jobs to apply business rule and to load into hdfs.• Recurring loads from the Oracle source system using an sFTP pull framework and Landing of files into the UNIX edge node.• Registration of the datasets in a metadata registry that controls admittance into Hadoop.• Designed criteria for progressive classification of the datasets within Hadoop (from raw to validated, refined and archived)• Designed the data processing approach within Hadoop using Pig.• Keying to insert keys to support the data lineage based on certain ids. This is done through a java program that looks up the data against a master file • Splitting to shard the dataset into NPI, Credit and Anonymous classifications as per federal regulations. This is accomplished through a python script where the schema of the split is passed as a parameter. • ELT, using Talend Big Data for processing the data. Transformations done include Tjoins, Treformats, Tsorts, if-else conditional transforms etc.

Cork, Ireland, Ie

• Develop the ETL solutions to migrate payroll from workday to ADP• Leading Development team, In Developing / DBA and Business to Develop the ETL solutions for various subject areas.• Designing the ETL process to load data from various different source to HDFS.• Interacting with Business/Domain to gather ETL requirements & analysis off various data sources, Flat files and oracle.• Provide Technical guidance and leadership on Etl methodologies for projects, such as technical requirements, source system analysis, Etl design topics, ETL control system, Etl Jobs, code reviews and testing procedures and Integration Testing.• Generating the Ad-hoc Report using the Hive.• Generating Report in Business Object as per the Business Requirement and resolving any data issue.• Worked extensively on Big Data, extensively used DataStage an ETL tool to design mappings to move data from Source to Oracle database-using Stages.• Interacting with DBA team to convert Logical Data Model into the Physical Data Model for HDFS recommending primary index and partition based on nature of query submitted by the end user.• Design and Development of Data stage ETL jobs using various stages to load Daily and Historical data into HDFS.• Involved in Data Extraction, Transformation and Loading (ETL process) from Source to target systems using Data Stage and Pig Scripts.• Successfully Integrated data across multiple source with high volumes of data.• Performance Tuning of existing Etl Jobs and Automation of ETL processes using DataStage Job Sequencer, Job Control routines and Transform functions. • Migration of Entire ETL jobs/Script Shell Scripts/DSN connection to production environment.

San Diego, California, Us

• Leading Development team, In Developing / DBA and Business to Develop the ETL solutions for various subject areas.• Extensively used Cloudera (Hive) to generate the Reports.• Provide Technical guidance and leadership on Etl methodologies for projects, such as technical requirements, source system analysis, Etl design topics, ETL control system, Etl Jobs, code reviews and testing procedures and Integration Testing.• Worked extensively on Big Data, extensively used DataStage an ETL tool to design mappings to move data from Source to Target database-using Stages.• Interacting with Business/Domain exert to gather ETL requirements & analysis off various data sources, Flat files, oracle, sql server, mainframe system.• Interacting with DBA team to convert Logical Data Model into the Physical Data Model and recommending primary index/secondary based on nature of query submitted by the end user.• Experience in Handling NLS Characters, i.e. loading CHN/JPN and Latin European language special characters.• Design and Development of Datastage ETL jobs using various stages like Aggregator, Join, Transformer, Sort, Merge, Filter and Lookup, Sequence, Modify, Peek etc. stages to load Daily and Historical data into star schema data model.• Created Generic CDC script in UNIX/Bteq to capture the new records, update and deletes.• Used Teradata utilities like Sql assistant, BTEQ, FASTLOAD and MLOAD.• Migration of Entire ETL jobs/script/bteq/Shell Scripts/DSN connection to production environment.

London, Gb

As an ETL developer, I am involved in • Designing the ETL jobs using DataStage 7.5.1 tool to load data from Mainframe to Teradata Data-base.• Created sequence for running ETLs using new job sequencer, job activity, nested condition, noti-fication activity and sequencer.• Used job sequence to create job sequencer, to call various parallel job and send message in case of process failures through the execution command.• Testing the Jobs and preparing the Unit Test Cases.

Bachelor'S Degree, Engineering/Industrial Management

Education record

Bachelor Of Engineering (B.E.), Industrial Production Engineering And Management