Krisha Kocherla Email & Phone Number

Toronto, On, Ca

• Led the development of C360 architecture, integrating data from multiple systems using MongoDB and Elasticsearch for real-time customer service prediction.• Orchestrated ADF pipelines for seamless data ingestion, transformation, and loading.• Oversaw data migration from on-prem SQL servers to Azure Databricks and Synapse Analytics, optimizing performance with partition tables.• Collaborated closely with ML and NLP modelers, working alongside data scientists on various projects to enhance data-driven insights.• Implemented data quality and error handling automation using shell scripts, reducing manual interventions and ensuring efficient data transitions across multiple systems.

• Developed a comprehensive data ingestion framework using Flask to manage ETL workflows from sources like MSSQL and Oracle, and integrated Snowflake with AWS Redshift for scalable cloud-based data warehousing. • Managed container orchestration with Kubernetes and Helm in RedHat OpenShift and optimized Azure Blob Storage and Databricks for secure data storage and large-scale processing. • Automated ETL pipelines in ADF, optimized AWS Redshift for faster queries, and improved performance in Azure Databricks using partitioned tables and parallel processing, while ensuring high availability with Kubernetes ConfigMaps and node selectors.

• Developed a data ingestion framework and ETL workflows using Spark-Scala to integrate various file systems into Hive and HDFS target tables.• Built a generic consumer engine using Spark for handling streaming data from Kafka, ensuring efficient loading into HDFS for further processing.• Created Spark DataFrames and Datasets to validate, transform, and process JSON and XML file formats, ensuring accurate data ingestion.• Utilized Oozie to automate and schedule Spark and Hive applications for large-scale data processing in HDFS.• Managed and monitored the Hadoop cluster using Ambari and Cloudera Manager, enhancing cluster performance and scalability.• Developed shell scripts to automate data validation, job scheduling, and movement between HDFS and Edge Nodes, significantly reducing manual effort.

• Hadoop Data Lake Development: Designed and developed a data lake using HDFS for storing, processing, and analyzing large-scale data from multiple sources in the Nielsen Analytics Platform (AP).• Spark & MapReduce Integration: Implemented Spark and MapReduce jobs for ETL processes, transforming data before loading into Hive and Impala for faster retrieval.• HiveQL Scripting & UDF Development: Created complex HiveQL scripts for data transformations, including joins, aggregations, and filters to process large datasets.• Workflow Automation with Oozie: Automated workflows using Oozie, ensuring smooth scheduling and execution of Spark and Hive jobs for efficient data processing.• REST API Development: Developed REST web services to handle communication between front-end systems and the Hadoop Common Platform, enabling live data search using ElasticSearch and Oracle.• Data Validation & Shell Scripting: Developed shell scripts to validate data, automate job execution, and manage file movement within the Hadoop Distributed File System (HDFS) and Edge Nodes.• CI/CD with Jenkins & Bitbucket: Implemented continuous integration using Jenkins and Bitbucket, managing code versions and automating deployments to various environments.