Krishnakanth B Email & Phone Number

East Brunswick, Nj, Us

Lewisville, Tx, Usa

 Integrated Hive and HBase for streamlined operations and data management. Developed Spark programs in Scala for data transformations, utilizing Datasets, Data Frames, and Spark p queries, including Spark Streaming and windowed streaming applications. Utilized Azure Databricks with PySpark to optimize data transformation speed and accuracy. Created on-demand tables from S3 files using Lambda functions and AWS Glue with Python and PySpark. Designed and implemented PyUnit framework for unit testing. Developed complex Transact-SQL queries and SSIS packages for data warehouse loading. Built Admin API for managing Kafka objects such as topics and brokers. Implemented data ingestion modules with AWS Glue, loading data into S3 layers and facilitating reporting via Athena and QuickSight.  Managed data storage on Azure Data Lake Store (ADLS) and Azure Blob Storage, ensuring data durability and accessibility.  Leveraged Kubernetes features like Ingress controllers, Persistent Volumes, and Secrets to configure networking, storage, and security policies for applications. Developed data quality scripts using SQL and Hive to validate data integrity post-load.  Integrated Azure Databricks with Azure Data Lake Analytics for advanced analytics. Utilized Python and Tableau to create various data visualizations. Proficiency in containerizing applications and services using Docker for easier deployment, scalability, and isolation of environments. Experience in defining and managing multi-container Docker applications using Docker Compose for local development and testing. Proficient in monitoring and managing Hadoop clusters through Cloudera.  Implemented data lineage and cataloging using Azure Purview.

Hyderabad, Telangana, India

 Evaluated business requirements and prepared detailed specifications in adherence to project guidelines for developing written programs. Integrated data storage solutions in Spark, particularly with Azure Data Lake Storage and Blob Snowflake Storage. Developed Spark programs in Scala for data transformations, creating Datasets, DataFrames, and executing Spark SQL queries, including Spark Streaming and windowed streaming applications. Utilized AWS Glue catalog with a crawler to extract data from S3 and perform SQL query operations. Extensively worked with Hive, including creating Hive tables and loading event data from Kafka using Spark Streaming. Understanding of Docker networking concepts to connect containers and manage data persistence using Docker volumes. Contributed to data migration from on-premises SQL Server to cloud databases (Azure Synapse Analytics (DW) & Azure SQL DB). Collaborated with DevOps team to integrate NIFI Pipeline on EC2 nodes with Spark, Kafka, and Postgres across QA and Production Environments using SSL handshakes. Developed Spark scripts and Spark SQL queries for data aggregation, querying, and integration with RDBMS via Sqoop. Managed copying Fact and Dimension data from S3 to Redshift for historical data analysis using Tableau and QuickSight. Designed and developed Admin API for managing Kafka topics, brokers, and other objects. Developed Producer and Consumer APIs for publishing and subscribing to event streams across one or more Kafka topics.