Harshith Reddy Kothapally Email & Phone Number

Albany, NY, US

New York, United States

• Designed and managed end-to-end data pipelines using Azure Data Factory (ADF), Azure Data Lake Storage Gen2 (ADLS Gen2), and Azure Databricks for efficient data handling, landing, and high-performance transformation.
• Performed scalable feature engineering, optimized data ingestion and storage across various formats (Avro, ORC, JSON, CSV, Parquet), and boosted processing speed with Kyro serialization in Spark.
• Utilized Matillion/Lakehouse architecture for efficient data handling, implemented Parquet with partitioning and Delta Cache for optimized access and reduced recomputation, and leveraged Azure Purview for streamlined.
• Automated data classification and labeling, promoted data discovery and governance best practices, and established robust data governance with Azure Active Directory (Azure AD).
• Leveraged vectorized UDFs, broadcast variables, and parallel operations for faster processing in Spark, monitored performance for resource optimization, and utilized Spark tools for efficient data preparation and.
• Designed data pipelines with Informatica IICS, developed custom activities and workflows with various tools like Azure Functions, Databricks, PowerShell, and Azure Logic Apps, established a single Source of Truth with.

Dallas, Texas, United States

• Built real-time data pipelines with Apache Kafka & Azure Event Hubs, handling high-volume data surges from diverse sources like network sensors and IoT devices. Guaranteed data integrity even during failures with.
• Performed real-time anomaly detection and data filtering using Azure Stream Analytics, identifying network issues and enabling swift fraud intervention. Delivered valuable insights through interactive Power BI.
• Developed Delta Lake pipelines with ACID transactions for data consistency and integrity. Managed Azure resources with ARM templates for efficient data management and implemented robust data governance using Azure.
• Enhanced query performance in Snowflake with materialized views. Implemented CRUD operations using Snowflake SQL for effective data manipulation.
• Designed and executed an end-to-end data migration pipeline for a modernized architecture, delivering cost optimization and deeper customer insights.
• Unearthed valuable insights from Telemetry and Timeseries data (IoT sources) using Jupyter Notebook. Improved performance and enabled predictive maintenance capabilities.

Boston, Massachusetts, United States

• Designed and implemented highly scalable Hadoop clusters (Hortonworks/Cloudera) for efficient large-scale data processing, enabling businesses of all sizes to analyze massive datasets smoothly.
• Built and automated robust data pipelines using Sqoop, Flume, NiFi, Airflow, and Oozie. These pipelines integrate diverse data sources, ensuring smooth and reliable data flow for timely insights.
• Championed real-time processing with Kafka and Spark Streaming, capturing near-instantaneous data changes and delivering actionable insights at the speed of business, empowering real-time decision-making.
• Mastered data transformation across diverse formats (Avro, JSON, Parquet) using Hive, Spark SQL, and custom UDFs. Performed complex data cleansing and transformations, ensuring data quality and analysis readiness.
• Optimized performance for exceptional speed, efficient resource utilization, and streamlined development through partitioning, bucketing, compression, and CI/CD practices.
• Integrated data pipelines with external systems and data warehouses using APIs and connectors, breaking down silos and creating a unified data landscape for comprehensive analysis, facilitating improved collaboration.

New York, New York, United States

• Developed data marts for downstream reports and built a User Access Tool using PL/SQL for ad-hoc analysis and query execution.
• Implemented efficient PL/SQL stored procedures for data manipulation, aggregation, and automation, with error handling and debugging for code quality and data integrity.
• Leveraged cursors effectively within stored procedures for intricate data retrieval and processing, tackling complex datasets seamlessly.
• Designed and implemented Star Schema and Snowflake Schema data models for efficient data warehousing and analysis, optimizing relationships and dimensional hierarchies.
• Utilized Visual Studio 2010 for data analysis, visualization, and troubleshooting, extracting insights from complex datasets and resolving software issues efficiently.
• Normalized dimension tables in Snowflake to minimize data redundancy and enhance data integrity.

Master'S Degree

Masters in Computer Science graduate with an in-depth understanding of advanced software engineering principles. Proficient in diverse.

Bachelor'S Degree, Computer Science

Computer Science graduate with a strong foundation in software engineering principles and a passion for coding and problem-solving..