Kiran Ch Email & Phone Number

United States

United States

• Architected, designed and developed a fully automated payment integrity solution using Spark and Hadoop.• Led the effort to develop the analytics pipeline in channelizing the Web APIs to the pharmaceutical companies that provide the medical data in desired formats. It covers 90% of the team’s revenue and is now being used extensively in other internal teams.• Utilized advanced algorithms such as K-means Clustering and Convolutional Neural Networks (CNNs) to analyze large datasets in… Show more • Architected, designed and developed a fully automated payment integrity solution using Spark and Hadoop.• Led the effort to develop the analytics pipeline in channelizing the Web APIs to the pharmaceutical companies that provide the medical data in desired formats. It covers 90% of the team’s revenue and is now being used extensively in other internal teams.• Utilized advanced algorithms such as K-means Clustering and Convolutional Neural Networks (CNNs) to analyze large datasets in healthcare and technology domains, optimizing ETL processes, automating data workflows, and enhancing overall efficiency and throughput.• Proficient in designing and implementing complex data transformation logic within DataStage, including data cleansing, enrichment, aggregation, and normalization techniques.• Experienced in leveraging DataStage's parallel processing capabilities to maximize job performance and scalability, including designing and implementing parallel job flows for optimal resource utilization.• Develop the Cassandra and SQL Lite integrated Java codebase which reads data from different source systems and transforms into customer required values and then pipeline it to multiple file formats as required by different customers.• Experience operationalizing NiFi on Kubernetes and AWS to transfer high volume real-time data between different sources and destination systems.• Experienced in leveraging DataStage's parallel processing capabilities to maximize job performance and scalability, including designing and implementing parallel job flows for optimal resource utilization.• Develop ELT data pipeline to migrate applications using DBT and Snowflake framework.• Collaborated with cross-functional teams to gather data requirements and translate them into technical specifications for Java-based data engineering tasks, showcasing excellent communication and problem-solving skills in Java development environments. Show less

Texas, United States

Create streams with Spark, process real-time data in RDDs and data frames, and create analytics with SPARK SQL.• Configure AWS infrastructure when implementing EC2 and S3 APIs to access S3 bucket data files.• Written ETL work on using Spark data pipelines to process data from different sources to transform data for multiple purposes.• Developed an end-to-end ETL pipeline using Spark SQL on Spark engine and imported data from AWS S3 to Spark RDD, performed transformations and actions… Show more Create streams with Spark, process real-time data in RDDs and data frames, and create analytics with SPARK SQL.• Configure AWS infrastructure when implementing EC2 and S3 APIs to access S3 bucket data files.• Written ETL work on using Spark data pipelines to process data from different sources to transform data for multiple purposes.• Developed an end-to-end ETL pipeline using Spark SQL on Spark engine and imported data from AWS S3 to Spark RDD, performed transformations and actions on RDD.• Experience working with the Spark ecosystem using Spark SQL queries in various formats such as text files and CSV files.• Expertise in utilizing Java frameworks and libraries such as Apache Hadoop, Apache Spark, and Apache Kafka to process and analyze large volumes of data in distributed computing environments.• Proficient in designing and implementing data ingestion pipelines using Java-based technologies, enabling seamless integration of diverse data sources into Big Data platforms.• Developed multiple POCs using PySpark Spark SQL module and branching using Hive Query Language (HQL).• Experienced in designing Star Schema, Snowflake schema for Data Warehouse, by using tools like Erwin data modeler, Power Designer and Embarcadero E-R Studio.• Load and transform large amounts of structured, semi-structured and unstructured data.• Use the Informatica MDM tool to continuously monitor master data to create reports and alerts to identify data quality issues and track data quality metrics.• Written SQL queries and stored procedures that allow applications to communicate with the database.• Design and implement Spark SQL tables and Hive script tasks to plan and create workflows and workflows.• Experienced in working with Data Governance, Data Quality and Metadata Management team to understand project.• Involved in ETL phase of the project, designed and analyzed data in Oracle and migrated to Hive.• Participate in customer meetings and clarify opinions on support. Show less

Cleburne, Texas, United States

• Designed and implemented data pipelines using Azure cloud platform (HDInsight, Data Lake, Data Bricks, Blob Storage, Data Factory, Synapse, SQL, SQL DB, DWH and Data Storage Explorer).• Developed a custom ETL, batch processing and real-time facts ingestion pipeline to transport facts to and from Hadoop cluster the usage of PySpark and shell scripts.• Integrating on-premises data (MySQL, Cassandra) with the cloud (Blob Storage, Azure SQL Database) and applied transformations to reload… Show more • Designed and implemented data pipelines using Azure cloud platform (HDInsight, Data Lake, Data Bricks, Blob Storage, Data Factory, Synapse, SQL, SQL DB, DWH and Data Storage Explorer).• Developed a custom ETL, batch processing and real-time facts ingestion pipeline to transport facts to and from Hadoop cluster the usage of PySpark and shell scripts.• Integrating on-premises data (MySQL, Cassandra) with the cloud (Blob Storage, Azure SQL Database) and applied transformations to reload into Azure Synapse using Azure Data Factory.• Docker container images built and published using the Azure Container Registry and deployed to Azure Kubernetes Service (AKS).• Experienced in CDC data using Spark and saved as Parquet in HDFS for later analysis.• Implemented a 'serverless' architecture using API Gateway, Lambda, and Dynamo DB and deployed AWS Lambda code from Amazon S3 buckets. Created a Lambda Deployment function, and configured it to receive events from your S3 bucket• Experienced in Data Warehousing Concepts like Ralph Kimball Methodology, Bill Inmon Methodology, OLAP, OLTP, Star Schema, Snowflake Schema, Fact Table and Dimension Table.• Configure Azure Data Factory (ADF) to ingest data from various sources such as relational and non-relational databases to meet business functionality requirements.• Streamlined workflows by creating DAGs in Apache Airflow to schedule ETL jobs and implementing additional components in Apache Airflow such as pools, executors and multi-node functionality.• Experienced in performing database migration from Sun Solaris to Linux, and from Linux to Oracle Cloud Infrastructure (OCI) using traditional data pump, Golden gate for zero downtime.• Improve airflow performance by reviewing and implementing the most appropriate configurations.• Design cloud architectures and implementation plans for web website hosting complicated programs on MS Azure.• Perform transformation level operations using Apache Spark RDD, Data Frame and SparkSQL Show less

India

Masters, Computer Science

Bachelor'S Degree, Electrical, Electronics And Communications Engineering