Chetan K Email & Phone Number

Irving, TX, US

As a Data Engineer completed End to End ETL pipeline using Python, Spark, SQL and AWS Cloud.

Tarrytown, New York, US

AWS Big Data Engineer

New York, New York, US

Data Engineer at Oscar Health.

Beaverton, OR, US

Works for Member Analytics Team using PySpark & AWS.

Remote, US

Developed ETL Data Pipelines using Databricks PySpark.

Seattle, WA, US

Chicago, Illinois, US

• Options Clearing Corporation(OCC) is one of the leading Options Clearing house in the United States for Options and Futures also. The Scope of the CST Internalization analytics project is to identify the credit cover.
• Responsible for Mapping of data before ingesting according to business problem.
• Responsible for ingesting large volumes of data into Spark Cluster from IBM DB2 databases using Queries. Also used HDFS, S3 along with IBM DB2.
• Developed Spark Script with PySpark, Java using PyCharm Spring Boot IDE that performs the internalization process.
• Worked on mainly developing Pyspark code using existing resources like QA code written in python, Hanweck BRD to eliminate the previous flaws in design along with performance improvement.
• Wrote Spark Dataframes, Datasets and RDD’s that uses mainly PSV files, Avro & parquet files format also. Used Spark SQL extensively.

Dublin 2, IE

• Accenture is one of the leading Consulting firms in the United States for software development. The Scope of the PiaB project is to build different microservices along with AWS cloud resources as support and Kafka is.
• Responsible for ingesting large volumes of IOT data to Kafka.
• Developed Microservices with Java using Spring Boot IDE.
• Worked on identifying present Scripted syntax Jenkins pipeline style and suggested to changing to Declarative style for reducing deployment time.
• Wrote Kafka producers to stream the data from external rest APIs to Kafka topics.
• Experience working for Security groups in AWS cloud and working with S3.

• Ulta Beauty is one of the leading Retailer in the United States of personal beauty products. The Scope of the project is to analyze user interactions with Ulta applications and facilitate Data Science team and Business.
• Responsible for ingesting large volumes of user behavioral data and customer profile data to Analytics Data store.
• Developed custom multi-threaded Java based ingestion jobs as well as Sqoop jobs for ingesting from FTP servers and data warehouses.
• Developed many Spark applications for performing data cleansing, event enrichment, data aggregation, de-normalization and data preparation needed for machine learning exercise.
• Worked on troubleshooting spark application to make them more error tolerant.
• Worked on fine-tuning spark applications to improve the over-all processing time for the pipelines.

• I worked mainly on data analysis using Hadoop tools.Responsibilities:
• Worked with the business team to gather the requirements and participated in the Agile planning meetings to finalize the scope of each development.
• Responsible for building scalable distributed data solutions on Cloudera distributed Hadoop.
• Developed multiple MapReduce jobs in Java for data cleaning and preprocessing.
• Implemented data pipelines developing multiple mappers by using Chained Mappers API.
• Developed multiple MapReduce batch jobs in java for loading the data to HDFS in sequential format.

• Responsibilities:
• Involved in Analysis, design and development of web applications based on J2EE.
• Struts framework is used for managing the navigation and page flow.
• Developed the EJB-Session Bean acts as Facade, will be able to access the business entities through their local home interfaces.
• Designed the user interface using HTML, CSS, java Script and JQuery
• Used Log4j to debug and generate new logs for the application.

Masters, Information Systems

Slcs Engineer College, Electrical, Electronics And Communications Engineering