Likitha Reddy Email & Phone Number

Mclean, Va, Us

Irving, Texas, United States

Currently working remotely as a Data Engineer• Responsible for design, development and delivery of data from Operational systems and files into ODS, downstream Data Marts and Files.• Developed Python scripts to find the vulnerabilities with SQL Queries by doing SQL injection, permission checks and analysis. Worked on complex SQL Queries, PL/SQL procedures and covert them to ETL tasks.• Involved in running Hadoop streaming jobs to process Terabytes of data.• Built the code efficiently and worked with business analyst, end users and architects.• Involved in scheduling Oozie workflow engine to run multiple Hive jobs.• Developed new ETL process in Spark and converted the existing Hive scripts into Spark.• Implemented custom UDF using Spark RDDs, Data frames, and Spark SQL.• Designed scalable data architectures for batch and real-time processing on AWS and GCP, capable of handling millions of records daily, optimizing system performance, and scalability. Developed and maintained ETL workflows using Apache Spark and AWS Glue, facilitating efficient data movement from on-premise systems to the cloud.•Designed the optimal performance strategy, and managed the technical metadata across all ETL jobs.• Implemented scripts in PySpark to validate the data and automated the scripts.• Involved on designing, developing, testing, tuning and building a large-scale data processing system.• Involved in data migration using NIFI.• Scheduled the jobs and Data-bricks work flows using Airflow.• Responsible for building solutions involving large data sets using SQL methodologies, Data Integration Tools like Informatica in any database.• Used JIRA for bug tracking and GIT for version control.• Followed agile methodology for the entire project.

Charlotte, North Carolina, United States

• Replaced the existing MapReduce programs and Hive Queries into Spark application using Scala.• Developed data pipeline using Flume, Sqoop and Pig to extract the data from weblogs and store in HDFS. • Loaded the data from Teradata to HDFS using Teradata Hadoop connectors. • Developed Shell scripts for scheduling and automating the job flow.• Performing Hive tuning techniques like partitioning and bucketing and memory optimization.• Hands on experience on Sqoop import, export and eval.• Worked on migrating MapReduce programs into Spark transformations using Spark and Scala, initially done using Python (PySpark).• Performed Distcp while loading the historic data in to Hive.• Used Spark SQL to load data and created schema RDD on top of that which loads into Hive tables and handled structured using Spark SQL.• Involved in converting the Hql into Spark transformations using Spark RDD with support of Python and Scala.• Performed Pig script which picks the data from one hdfs path and performs aggregation and loads into another path which later pulls populates into another domain table. Converted this script into a jar and passed as parameter in Oozie script.• Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the SQL Activity.• Build an ETL which utilizes spark jar inside which executes the business analytical model.• Hands on experiences on Git bash commands like git pull to pull the code from source and developing it as per the requirements, Git add to add files, git commit after the code build and git push to the pre prod environment for the code review and later used screwdriver. Yaml which build the code, generates artifacts which releases into production.• Performed data validation which does the record wise counts between the source and destination.• Have knowledge on apache Hue.• Good hands-on experience with Git and GitHub.• Created a feature node on GitHub.