Harshitha D Email & Phone Number

New York, New York, United States

Developed scalable and cost-effective AWS Big Data architecture for data lifecycle management including collection, ingestion, storage, processing, and visualization of large, rapidly changing datasets. Implemented end-to-end data pipelines using PySpark for data ingestion into HIVE tables, performing data cleansing, aggregation, and de-duplication. Designed and deployed multi-tier applications using AWS services like EC2, Route53, S3, RDS, DynamoDB, SNS, SQS, IAM with a focus on high availability and fault tolerance.Leveraged Spark features for efficient data preprocessing, scheduling jobs with Airflow, and writing unit tests for Spark code. Managed data extraction from multiple sources, Glue Catalog creation, and Lambda-based automated data transformation. Automated job scheduling with UNIX shell scripts and Cron jobs. Utilized Oozie for data loading into HDFS, Spark streaming with Kafka, and various Spark transformations for data cleansing. Used Jira for issue tracking, Jenkins for CI/CD, and Glue for merging data from multiple tables. Implemented monitors, alarms, and logs using CloudWatch. Assessed AWS services like Amazon EMR, Redshift, and S3 for end-to-end architecture and implementation. Mentored analysts on building analytics tables and collaborated with DevOps for NIFI Pipeline integration with Spark, Kafka, and Postgres.

Hyderabad, Telangana, India

Implemented solutions leveraging advanced AWS components such as EMR and EC2, integrating with Big Data/Hadoop Distribution Frameworks like Hadoop YARN, MapReduce, Spark, and Hive. Developed a comprehensive cloud migration strategy and implemented best practices using AWS services including Database Migration Service and AWS Server Migration Service for seamless transition from On-Premises to the cloud. Enhanced Spark performance and optimization of existing algorithms in Hadoop using Spark Context, Spark SQL, Data Frame, and Spark Yarn. Managed daily backups of AWS PostgreSQL to S3 through EMR using DataFrames. Authored BASH and Python scripts for AWS infrastructure creation, utilizing Lambda functions for code upload and monitoring changes in S3 and DynamoDB tables. Gathered data and business requirements, designing and building data solutions for migrating existing source data to Atlas Data Lake. Wrote Spark applications for data validation, cleansing, transformations, and custom aggregations, handling workload management for load balancing and failover. Created multiple data pipelines for various tasks including data conversion, analysis, visualization, and database migration. Demonstrated proficiency in Python and Scala, creating user-defined functions for Hive and Pig. Utilized AWS Data Migration Service for Aurora RDS migration, implemented Spark Streaming for near-real-time data processing, and managed different file formats and compression techniques. Orchestrated CI/CD processes using CloudFormation, Terraform, and Docker across multiple environments. Developed shell scripts for dynamic partitions, schema verification, and Kubernetes for CI/CD runtime environment. Administered Jenkins for managing Build, Test, and Deploy processes and utilized GitHub for code control. Contributed to the transformation of departmental projects using Agile methodology and cloud platforms like Azure and AWS, facilitated by Jira.

Hyderabad, Telangana, India

Configured Spark Streaming to receive real-time data from Kafka and stored it in HDFS, converting unstructured data to structured data using Spark and proposing NoSQL solutions.Designed and built modern data solutions on Azure PaaS for data visualization, considering existing production states and impact on business processes. Utilized Azure services including Data Factory, T-SQL, Spark SQL, and Azure Databricks for ETL processes and data processing. Deployed, configured, and maintained compute resources on Azure Cloud, focusing on Azure SQL Database, Analysis Service, Data Warehouse, Data Factory, and SQL Data Warehouse administration. Developed Spark applications for data extraction, transformation, and aggregation from various file formats. Optimized Spark job performance and reduced cluster run time to save costs. Designed solutions using Azure Stream Analytics, Event Hub, and Service Bus Queue for processing real-time data. Collaborated with stakeholders to integrate Azure Active Directory authentication and improved scalability of distributed data solutions. Utilized configuration management tools like GitHub and Bamboo for CI/CD processes. Contributed to the development of machine learning libraries and implemented MapReduce jobs for data cleaning and preprocessing.

Bengaluru, Karnataka, India

Identified and documented accurate business rules and use cases through thorough requirements analysis. Investigated and resolved data flow integrity issues during data loading into target databases, ensuring a smooth and precise process. Developed and managed application SQL, PL/SQL stored procedures, triggers, partitions, primary keys, various indexes, constraints, and views. Drafted and enhanced SQL scripts to evaluate the flow of online quotes into the database, guaranteeing data validity. Utilized historical data to evaluate and validate the efficacy of data prediction systems. Analyzed transactions to establish a cohesive business intelligence model, addressing real-time reporting needs. Executed SQL queries, integrating them with business intelligence (BI) tools such as Jupyter Notebooks (Python), Tableau Dashboards, and Excel reports. Extracted and interpreted data patterns, converting insights into actionable outcomes. Designed and implemented interactive dashboards featuring a variety of charts for easy comprehension. Produced multiple Excel documents to gather metrics data and presented information to stakeholders, offering clear explanations for optimal resource allocation. Analyzed datasets sourced from over 20 APIs, creating reports to highlight key data trends for the leadership team. Implemented various Tableau features such as joins, relationships, data blending, calculated fields, Level-of-Detail (LOD) expressions, parameters, hierarchies, sorting, groups, and actions (filter, highlight, and URL) to enhance dashboard functionality.