V Satya Sai Dasari Email & Phone Number

Bellevue, Wa, Us

United States

• Experienced in Agile sprint backlog grooming, sprint planning, and conducting sprint reviews and retrospectives to ensure continuous improvement and efficient project delivery.• Designed and developed scalable and cost-effective architectures using AWS Big Data services, covering the entire data lifecycle from collection and ingestion to storage, processing, and visualization.• Created a PySpark-based Data Ingestion framework to import source claims data into HIVE tables, incorporating data cleansing, aggregation, and de-duplication logic to ensure updated and accurate records.• Developed an End-to-End data pipeline in a distributed environment utilizing Big Data tools, Spark framework, and Tableau for comprehensive data visualization.• Utilized Druid to perform real-time analytics on streaming data from diverse sources, delivering timely insights into dynamic datasets.• Enhanced Druid's real-time streaming capabilities, enabling the processing and analysis of high-velocity, high-volume data streams in real-time, thereby improving decision-making processes.• Conducted performance tuning and optimization of DRUID configurations, facilitating scalability and maximizing throughput within Apache Druid clusters.• Configured and fine-tuned DRUID components like Middle Managers and Historical for efficient data ingestion and improved query response times, enhancing overall system performance.• Responsible for continuous monitoring and managing Elastic MapReduce (EMR) cluster through AWS console. • Developed Spark applications using Scala and Spark-SQL for data extraction, transformation, and aggregation from multiple data file formats to uncover insights into the customer usage patterns.• Worked on processing batch and real time data using Spark using Scala.• Designed and Developed Real time Stream processing Application using Spark, Kafka, Scala, and Hive to perform Streaming ETL and apply Machine Learning.

United States

• Designed and implemented a user interface for B2B bid monitoring across various stages, leveraging TypeScript to improve operational oversight and stakeholder interaction.• Created data pipeline for different events of ingestion, aggregation, and load consumer response data into S3 buckets to serve as feed for AWS Quick sight Dashboard.• Developed test frameworks for validating data at various points within the data pipeline.• Implemented AWS Lambda functions to run scripts in response to events in Amazon DynamoDB table or S3 bucket or to HTTP requests using Amazon API gateway.• Implemented a serverless architecture using API Gateway, Lambda, SQS and Dynamo DB and deployed AWS Lambda code from Amazon S3 buckets. Created a Lambda function and configured it to receive events from S3 on every new file uploaded to the bucket.• Utilized AWS Step functions, AWS Lambda, PySpark to build a file check pipeline to parse the raw transaction files.• Designed and implemented scalable data pipelines using AWS Glue for ETL processes, integrating data from multiple sources into a centralized data warehouse.• Leveraged Amazon Redshift for data warehousing, performing complex SQL queries and optimizing performance for large-scale data analytics.• Designed and developed ETL pipelines and dashboards using Step Function, Lambda, EMR, CloudWatch, Glue and QuickSight.• Built real-time data processing applications using Amazon Kinesis, ensuring low-latency data streaming and efficient event-driven architecture.• Defined and deployed monitoring logs, alerts on CloudWatch to track data pipeline performance and troubleshoot issues.• Developed and integrated end-to-end data pipeline on AWS environment using AWS S3, PySpark to extract the data files and load into S3 bucket, reduced the processing time by 40%.• Used linked lists for managing customer data updates and binary trees for quick searching and sorting of customer records.

• Optimized DataStage parallel jobs to take full advantage of multi-threaded processing, improving job performance and reducing execution time.• Designed, developed, and maintained complex data workflows in Apache Airflow to automate ETL and data processing.• Worked with Cloud Data Catalog and other google cloud APIs for monitoring, query, and billing related analysis for BigQuery usage.• Automated ETL workflows using Google Cloud Composer (Apache Airflow), orchestrating data movement and transformation across various GCP services.• Creating alerting policies for Cloud Composer, Cloud Data fusion to notify on any job failure.• Managed and optimized data processing workflows using Google Cloud Platform (GCP) services including Dataproc, Google Cloud Storage (GCS), Cloud Functions, and Big Query, ensuring seamless integration and efficient data handling across the entire pipeline.• Built data pipelines in Airflow/Composer to orchestrate ETL jobs using different operators.• Integrated Git repositories with Google Cloud Source Repositories for version control of code deployed on GCP.• Utilized Google Cloud Pub/Sub for real-time messaging and event-driven architectures, ensuring low-latency data streaming and processing.• Optimized performance of data processing pipelines and queries on GCP by fine-tuning resource allocation, query optimization, and index management.• Conducted capacity planning and resource optimization exercises on GCP to right-size infrastructure and ensure optimal performance and cost-efficiency of data processing workloads.• Developed custom monitoring and alerting solutions on GCP using Stackdriver Monitoring and Logging, enabling proactive detection and resolution of performance bottlenecks and system failures.• Implemented data privacy and compliance measures on GCP using tools like Data Loss Prevention (DLP) API and Cloud Security Scanner, ensuring adherence to data protection regulations and industry standards.

Hyderabad, Telangana, India

• Involved in implementation of new statistical algorithms and operators on Hadoop and SQL platforms and utilized optimizations techniques, linear regressions, K-means clustering, Native Bayes and other approaches.• Developed Spark batch job to automate creation/metadata update of external Hive table created on top of datasets residing in HDFS.• Developed Data Serialization spark common module for converting Complex objects into sequence bits by using AVRO, PARQUET, JSON, CSV formats.• Worked on ERModeling, Dimensional Modelling (Star Schema, Snowflake Schema), Data warehousing and OLAP tools.• Populated HDFS and PostgreSQL with huge amounts of data using Apache Kafka.• Design and develop Rest API (Commerce API) which provides functionality to connect to the PostgreSQL through Java services.• Created iterative macro in Alteryx to send Json request and download Json response from webservice and analyze the response data.• Designed Batch Audit Process in batch\shell script to monitor each ETL job along with reporting status which includes table name, start and finish time, number of rows loaded, status, etc.• Developed Spark jobs in PySpark to perform ETL from SQL Server to Hadoop.• Proficient in creating complex Directed Acyclic Graphs (DAGs) using Apache Airflow for efficient job scheduling and monitoring.• Designed and implemented data acquisition, ingestion, Management of Hadoop infrastructure and other Analytics tools (Splunk, Tableau).• Developed python scripts to automate data ingestion pipeline for multiple data sources and deployed Apache Nifi in AWS.• Encoded and decoded JSON objects using PySpark to create and modify Data Frames in Apache Spark.• Designed and developed ETL mappings to collect data from various sources using REST API.• Led the migration of on-premises data infrastructure to GCP, overseeing assessment, planning, and execution phases, which improved scalability, reliability, and cost-effectiveness.

Hyderabad, Telangana, India

• Designed and implemented data warehouse structures, including the creation of fact tables, dimension tables, and aggregate tables using dimensional modeling techniques, specifically Star and Snowflake schemas.• Involved in converting existing Hive/SQL queries into Spark transformations using Spark RDD and Python.• Developed and executed data quality scripts using SQL and Hive to ensure accurate data loading and maintaining high standards.• Developed and optimized MapReduce jobs to process large datasets efficiently in a distributed Hadoop environment.• Created AWS Pipelines by extracting customer data from various sources such as Excel, Flat Files, Oracle, SQL Server, Teradata, and server log data into Hadoop HDFS.• Developed a python script to validate files which are being uploaded across several S3 buckets.• Designed and implemented scalable data architectures, including data lakes and data warehouses, to support the organization's data-driven initiatives.• Developed stored procedures in MS SQL to fetch the data from different servers using FTP and processed these files to update the tables.• Analyze various types of raw file like Json, Csv, Xml with Python using Pandas, NumPy etc.• Worked in Agile environment and used rally tool to maintain the user stories and tasks.• Conducted performance tuning and troubleshooting of ETL processes to enhance data processing efficiency and reliability.• Migrated existing QlikView dashboards to Power BI, ensuring high-quality data representation and preserving all key insights as per business needs. • Crafted interactive reports aligning with business objectives in Tableau & QlikView, improving data visualization and client insights.

Hyderabad, Telangana, India

• Developed ETL pipelines to extract, transform, and load data in and out of the data warehouse using a combination of Python and Snowflake's SnowSQL.• Designed and implemented 3NF data models for OLTP systems and dimensional data models using Star and Snowflake schemas for efficient data organization and retrieval.• Developed PySpark pipelines which transform the raw data from several formats to parquet files for consumption by downstream systems.• Leveraged Kafka for real-time data import into Hadoop and implemented Oozie workflows to schedule and manage data imports efficiently.• Developed a data pipeline using Spark Streaming and Kafka to store data in HDFS and perform real-time analytics on incoming data.• Orchestrated data preparation using Azure Data Factory and Databricks and loaded the processed data into SQL Data Warehouse.• Developed CI/CD pipelines and developed require docker images for the pipelines.• Developed ETL workflows using Informatica and SQL Server Integration Services (SSIS) to automate data extraction, transformation, and loading from diverse sources into data warehouses.• Developed pipelines for batch processing and scheduling using Azure Data Factory.• Used Azure Data Factory to orchestrate Databricks data preparation and load them into SQL Data warehouses.• Documented business workflows for stakeholder review.• Worked on End-to-End Process automation of enrolling new businesses using the Service Catalog tool.• Hands on experience in performing manual testing, regression testing and sanity testing.• Involved in creating the deployment packages and transmitted/deployed the packages across different environments like DEV, STAGE and PROD.

Master Of Engineering - Meng, Computer Engineering, 3.25

Bachelor Of Technology - Btech, Electrical And Electronics Engineering, 3.74