Yashwanth Reddy Email and Phone Number

As a Senior Data Engineer, I have extensive hands-on experience designing and implementing scalable data architectures, focusing on Snowflake for integrating and migrating legacy systems. I develop ETL pipelines using PySpark, Databricks, and AWS services like EMR, ECS, Glue, and Athena. I build advanced financial reports using SQL and DBT, utilizing Airflow for scheduling. I’ve led successful data migration projects across platforms like IBM DB2, Snowflake, and AWS Redshift, while also developing serverless architectures using AWS Lambda. Proficient in building microservices and RESTful APIs with Go, ensuring efficient memory management and high throughput for large-scale applications.My expertise includes creating scalable NoSQL databases in MongoDB, building RESTful APIs with Django, and managing data pipelines using Databricks and AWS Glue. Expert in developing highly concurrent and scalable backend services using Go, optimizing performance and reducing latency in distributed systems.Additionally, I optimize workflows, implement monitoring with Cloudwatch, and containerize applications with Docker to ensure efficient performance and scalability.

As a Data Engineer in Freddie Mac ,With extensive experience in cloud-based data migrations, I have successfully migrated data from S3 to both Azure Synapse Analytics (DW) and Snowflake. I am proficient in designing and managing data pipelines, scheduling triggers, and mapping data flows using Azure Data Factory (V2), while leveraging Key Vaults for secure credential management. I've worked with Azure Logic Apps and the ISE environment, using Kusto Explorer with KQL for log analytics and query optimization. In AWS, I have engineered solutions using Redshift, Glue, RDS, S3, and Lambda, writing PySpark and Python scripts to streamline data management and automate processes. Additionally, I have extensive experience with Azure Functions, Databricks, and have integrated various services like Azure Synapse, Step Functions, and API Gateway for scalable, event-driven architectures.

At Comcast ,I have hands-on experience developing RESTful Microservices with Python and Flask, optimizing AWS serverless Lambda functions with multithreading for concurrency, and working extensively with various AWS services like EC2, S3, RDS, VPC, IAM, Elastic Load Balancing, and CloudWatch. You've designed ETL pipelines using AWS Glue and Redshift, migrated pandas code to PySpark, and automated tasks with Python and Shell scripting. Additionally, you have expertise in Spark, Hadoop cluster tuning, PostgreSQL database management, and ensuring data security with robust encryption and authentication mechanisms.

At Barclays , I have hands-on experience designing and implementing end-to-end ETL pipelines on both on-premises Hadoop clusters and Azure cloud, utilizing Apache Hadoop tools, Azure storage solutions, and data warehousing solutions such as Azure Synapse and Azure SQL Database. Proficient in integrating CI/CD pipelines with tools like Kafka, Jenkins, Docker, and Kubernetes, they have implemented security measures to ensure compliance with GDPR, CCPA, and PCI DSS. With expertise in Hadoop ecosystem components (HDFS, YARN, MapReduce, Hive, Spark, HBase), they optimized Spark code with Scala and Spark-SQL, and utilized Informatica PowerCenter for ETL processes, ensuring data quality and reliability.