Pradeep Kumar Y Email and Phone Number

 Worked on creating a centralized data repository by building various ETL pipelines using Spark to process high-volume and real-time data using AWS Services. Developed code using Java/Scala for batch applications to run on Apache Spark Engine. Worked on Kafka Connect . Enhance a existing kafka framework to pull the live streaming data from various source systems. Developed a Data Ingestion Framework which loads data from various kafka streams and loads the data into AWS S3… Show more  Worked on creating a centralized data repository by building various ETL pipelines using Spark to process high-volume and real-time data using AWS Services. Developed code using Java/Scala for batch applications to run on Apache Spark Engine. Worked on Kafka Connect . Enhance a existing kafka framework to pull the live streaming data from various source systems. Developed a Data Ingestion Framework which loads data from various kafka streams and loads the data into AWS S3 and Snowflake data warehouse using Java, Scala, Spark, AWS SDK, Snowflake.  Developed a Tokenization Framework to handle the sensitive data coming from various sources to maintain data security across different stages of ETL pipeline using Java, Scala and Spark along with AWS. Developed a Data Quality check framework to maintain data integrity across the pipeline Show less

•Develop software related to asset management•Write and implement software solutions that integrate different systems•Write reusable and testable code•Integrate data storage solutions•Implement data security and protection