Deepthi N Email & Phone Number

United States

Louisville, Kentucky, United States

• Design and develop ETL processes in AWS Glue to migrate payments data from external sources like S3, Parquet/Text files into AWS Redshift.• Implemented DBT (Data Build Tool) in an AWS environment to streamline and automate data transformation processes, enabling efficient and scalable data pipeline management for analytics and reporting.• Created End-End Automation with CI Procedures using Jenkins & automated Maven builds by integrating them with Continuous Integration tools Jenkins • Extract Transform and Load data from Sources Systems to AWS Storage services using a combination of AWS Glue, Apache MiFi.• Implemented end-to-end data ingestion, enrichment, and accessibility solution on AWS, utilizing services such as AWS Glue, AWS Lambda, Amazon Redshift• Designed and developed data ingestion pipelines using Snowflake's features, including bulk loading and streaming, to seamlessly integrate data from various sources into the Snowflake platform.• Integrated Kafka & Flink with other tools like Hadoop, Spark, and Elastic search to enable real-time data analytics and insights.• Designed infrastructure for AWS application and workflow using Terraform and had done implementation and continuous delivery of AWS infrastructure using Terraform.• Developed Python and PySpark code for ETL and used them in AWS services like AWS Lambda, Glue, Ec2, EMR, etc.• Developed a shell script to run the ETL jobs from Control-M job scheduler• Hands on experience in migrating on premise data to Google Cloud Platform (GCP) using cloud native tools such as Big Query, Cloud Data Proc, Google Cloud Storage, Composer, Pub/Sub & Cloud Dataflow.• Implemented real time data streaming pipeline using Athena, AWS Kinesis, Airflow, Dynamo DB and deployed AWS Lambda code from Amazon S3 buckets.• Extensive experience in designing and implementing NoSQL databases like MongoDB and Cassandra for high volume data storage and retrieval.

Libertyville, Illinois, United States

• Involved in the project life cycle including design, development and implementation and verification and validation • Extensively utilized Python frameworks like Django, Flask, PyUnit, and libraries like matplotlib. • Gathered design requirements from the client, analyzed, and provided solutions and met design requirements.• Updated and manipulated content and files by using python scripts.• Created logic using Python programming language to node, functions and pipelines to retrieve and manipulate data.• Implemented Pandas, Pandas Data Frame query, NumPy Python libraries, performed complex table joins, extractions and transformation of financial data retrieved from NetSuite and migrated to SAP.• Implemented ETL pipeline to source data files and cross reference tables from AWS S3 and perform data conversion using logic implemented in Python.• Copied data from AWS S3 into Redshift using SQL and performed text mining and analysis using DBeaver.• Fetched twitter feeds for certain important keyword using the python-twitter library (Tweepy), Used JSON to store the twitter data which was further represented using matplotlib visualization and generated graphical reports for business decision using matplotlib library. • Wrote queries using SQL in DBeaver to load data from AWS S3, transfer into AWS Redshift and perform transformation.• Used Spark (PySpark SQL) in Spark Standalone mode and managed constant data changes to data analysis and migration. • Participated in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings with various business users.

Cincinnati, Ohio, United States

• Used Pandas API to put the data as time series and tabular format for east timestamp data manipulation and retrieval.• Worked on Data mapping, logical data modeling, created class diagrams and ER diagrams and used SQL queries to filter data within the Oracle database.• Created a user review data classification pipeline using Apache Spark and Apache Airflow. • Created a data extraction pipeline for migrating user review data from PostgreSQL dB to AWS Redshift. • Deployed AWS Lambda functions to ingest user purchase data on a daily basis and stage it in S3. • Designed a REST API backend using Django to provide access to marketplace trends dataset, this was used by product management team.• Wrote and executed various MYSQL database queries from Python using Python-MySQL connector and MySQL dB package• Used AWS lambda to run servers without managing them and to trigger to run code by S3 and SNS.• Setup storage and data analysis tools in Amazon Web Services (AWS) cloud computing infrastructure.• Updated Python scripts to match training data with our database stored in AWS Cloud Search, so that we would be able to assign each document a response label for further classification. • Performed troubleshooting, fixed and deployed many Python bug fixes of the two main applications that were a main source of data for both customers and internal customer service team.• Used Python scripts to update content in the database and manipulate files.• Generated Python Django Forms to record data of online users

Portland, Oregon, United States

• Analyzed pre-existing predictive model developed by advanced analytics team and factors considered during model development. • Analyzed metadata and processed data to get better insights of the data.• Analyze and prepare data, identify the patterns on dataset by applying historical models.• Designed and developed data management system using MySQL. • Built application logic using Python.• Angular.js is used to build efficient backend for client web application.• Used Python to extract information from XML files.• Worked on development of SQL and stored procedures on MYSQL.• Designed and developed a horizontally scalable APIs using Python Flask.• Used supervised machine learning techniques such as Logistics Regression and Decision Tree Classification. • Worked on datasets containing large values which are structured and unstructured data.• Performed Data cleaning process- Forward filling methods on dataset for handling missing values. • Created initial data visualizations in tableau to provide basic insights of data to the project stakeholders. • Performed extensive exploratory data analysis using Teradata to improve the quality of the datasets• Experienced in various Python libraries like Pandas, One dimensional NumPy and Two dimensional NumPy. • Worked for the Analytics team, to update the regular reports and providing solutions.• Creating visualizations for the data extracted with the help of Tableau.• Identifying patterns and meaningful insights from data by analyzing it.• Twisting SQL queries for improving performances.• Performing Data Modeling in Tableau Desktop.

Boston, Massachusetts, United States

• Writing Python scripts to parse XML documents as well as JSON based REST Web services and load the data in database.• data and for building the pipeline.• Experienced in the conversion of unstructured data in the format of CSV, and JSON into structured data in parquet form and saved in AWS s3 buckets.• Experienced in the creation of Data zones that included AWS Glue databases and AWS Glue catalog that help in giving easy access to business and Tableau connection setup.• Performed SQL optimization for the creation of AWS Views in Glue DB.• Writing ORM’s for generating complex SQL queries and building reusable code and libraries in Python for future use.• Working closely with software developers and debug software and system problems.• Profiling Python code for optimization and memory management and implementing multithreading functionality.• Involved in creating stored procedures that get the data and help analysts to spot the trends.• Designed and developed the server module and resolved issues and responsible for its enhancements.• Worked on Django ORM module for signing complex queries.• Used WPF to create user interfaces for Windows operating system.• Implemented business logic in Python to prevent, detect and claim duplicate payments Rewrite existing Python/Django modules to deliver certain format of data.• Used Django Database APIs to access database objects.