Charitha Reddy K Email & Phone Number

Clearwater, Florida, United States

In this project, I was responsible for architecting and deploying scalable data pipelines using AWS serverless technologies, which significantly reduced operational overhead and ensured data integrity and security throughout the data lifecycle. I played a crucial role in migrating existing MapReduce programs to Spark, utilizing PySpark and Spark SQL for more efficient data processing. My work with Snowflake involved creating and optimizing data pipelines, enabling the seamless flow of data from diverse sources, including cloud storage, on-premises databases, and flat files, into our data lake and data warehouse. I also integrated Snowflake with BI tools like Tableau and Power BI, facilitating advanced data visualization and reporting.I developed and optimized ETL processes using Databricks, leveraging Spark’s distributed computing capabilities. Implemented medium to large scale BI solutions on Azure using Azure Data Platform services (Azure Data Lake, Data Factory, Stream Analytics, Azure SQL DW, HDInsight/Databricks, NoSQL DB). Additionally, I migrated on-premises applications to AWS, using services such as EC2, S3, and EMR. I also implemented real-time data processing using Kafka and contributed to the documentation of AWS architectures and operational procedures.

Beaverton, Oregon, United States

In this project, I developed and optimized data solutions, including crafting high-performance Spark streaming applications to ingest data from cloud sources into Hive tables. I used Scala to create microservices and backend APIs with Akka and Play frameworks, and designed Scala workflows for efficient data extraction and transformation. I implemented robust data security measures in Snowflake and Kafka, focusing on encryption, access controls, and performance optimization.I designed a STAR schema data warehouse, built interactive Tableau dashboards for real-time data visualization, and automated ETL pipelines using Apache Airflow with customized DAGs. Leveraging AWS services, I enhanced data architecture and analytics capabilities. Demonstrated proficiency in infrastructure as code (IaC) tools such as Terraform, scripting in languages like Python and Shell, and experience with container orchestration tools, particularly Kubernetes on GCP. I also converted SQL queries into Spark RDD transformations, developed Hive tables and queries for complex data processing, and wrote MapReduce programs for various file formats. My role included handling complex SQL, PL/SQL, and MongoDB tasks while following Agile methodologies.

Oakland, California, United States

In this project, I gathered business requirements and designed data products while developing Spark applications for large-scale data transformations and denormalization. I ensured Kafka clusters' fault tolerance through replication and leader election, and wrote Spark programs in Scala for data quality checks. I utilized Python shell commands for Spark scripts and crafted Pig and Hive scripts with UDFs for ETL on AWS Cloud Services.Worked on Azure Data Factory to configure data loads from Azure Blob Storage into Azure Synapse Analytics. Utilized Azure Databricks, Spark, and other big data processing frameworks on Azure HDInsight clusters to process large datasets efficiently. Integrated HDInsight clusters with other Azure services such as Azure Blob Storage for data storage and Azure Data Lake Storage Gen2 for ETL (Extract, Transform, Load) jobs. I optimized Hive tables and queries, and implemented Terraform scripts for automating AWS infrastructure management, including ELB, CloudFront, RDS, and more. I imported data from AWS S3 into Spark RDDs, performed transformations, and edited SQL queries in Tableau for dynamic and static datasets. Additionally, I worked with NoSQL databases like HBase, creating tables for semi-structured data, and participated in SCRUM meetings to drive project progress and productivity.

New York, United States

I have led the development of complex Python and Scala code to fulfill application requirements for data processing and analytics. My responsibilities included creating and optimizing Spark code in Python and SparkSQL for efficient in-memory computations and resource management. I managed AWS costs effectively by optimizing resource usage and tracking expenses.I documented Kafka cluster configurations and data flows, developed custom PIG UDFs and loaders, and designed functional Tableau reports based on user needs. My role involved developing ETLs with Spark SQL, RDDs, and Data Frames, and performing text analytics using Scala within Spark.I also migrated MapReduce programs to Spark transformations, managed diverse data feeds such as JSON, CSV, and XML, and implemented the Data Lake concept. Additionally, I conducted data analysis with SQL queries and adhered to Agile practices, actively participating in SCRUM meetings and sprint activities.

Chennai, Tamil Nadu, India

I have led the development of Spark programs using Python APIs to import and transform data from S3 into Spark Dynamic Frames and DataFrames. I utilized Spark core and Spark SQL libraries for efficient data processing and performed advanced data transformations.In addition to developing complex Python and Scala code for data processing and analytics, I worked on web development with Scala, creating applications and APIs using the Play Framework. I developed Pig scripts for change data capture and delta processing in HDFS and integrated data quality plans into ETL processes.My role also included building key business metrics, visualizations, dashboards, and reports with Tableau, writing Hive queries for data analysis, and handling various file formats like JSON, CSV, and text. I created and modified SQL queries, prepared statements, and stored procedures, and actively participated in status meetings to provide updates to the management team.