Varshith Reddy Email & Phone Number

United States

San Antonio, Texas, United States

• Created Glue ETL jobs which reads the data from S3 and process the data using transformation techniques and stores the data in s3.
• Created Glue Crawlers which crawl the data present in s3 and create the Glue tables.
• Used Athena service to query the above tables.
• Implemented a serverless architecture using API Gateway, Lambda and DynamoDB.
• Created lambda functions and configured it to receive events from s3 bucket.
• Designed data models that are used in AWS lambda applications which are aimed to do complex analysis creating analytical reports for end-to-end traceability, lineage, definition ok key Business elements from Aurora.

Liberty, New York, United States

• Analyzed SQL scripts and designed the solution to implement using Scala.
• Developed analytical component using Scala, Spark, and Spark Stream.
• Performed advanced procedures like text analytics and processing, using the in-memory computing capabilities of Spark using Scala.
• Developed Spark code to using Scala and Spark-SQL for faster processing and testing.
• Designed scalable (Scala) Web Architecture hosting reports for the entire application.
• Wrote coding highly flexible, scalable, & distributed applications using Scala.

Delhi, India

• Spent a lot of time working with Hive's metadata imports and migrating existing tables and programs to run on Hive and the AWS cloud.
• Created a standard framework to export data using Sqoop to external data bases and import data from external data bases into HDFS.
• Worked on integrating EMR with Spark 2, S3, and Snowflake on Amazon Web Service (AWS)..
• Accountable for operating the Elastic MapReduce cluster through the AWS UI and continuously monitoring it.
• Used Spark Data Frame Operations to perform required Validations in the data and to perform analytics on the Hive data
• Developed PySpark scripts, using both Data Frames/SQL and RDD in Spark for data aggregation and queries.

Mumbai, Maharashtra, India

• Accountable for developing Hadoop-based scalable distributed data solutions. Hadoop/HBase clusters were set up and benchmarked for internal use.
• Java programming language was used to create simple to complicated Map/Reduce jobs, which were then implemented using Hive and Pig.
• Participated in all phases of data mining, data collection, data cleaning, developing models, validation, visualization, performed Gap analysis to provide feedback to the business team to improve the software delivery.
• Creating reusable Python scripts to ensure data integrity between the source (Teradata/Oracle) and target system.
• Continuous monitoring and managing the Hadoop cluster using Cloudera Manager.
• Developed programs in Spark to use on the application for faster data processing than standard MapReduce programs