Sai Teja Reddy Email & Phone Number

Dallas, Texas, United States

At Capital One, I worked as a GCP Data Engineer on the Integrated Credit Risk Analytics Platform, a key initiative aimed at improving credit assessment processes through the application of advanced analytics and machine learning. In this role, I was responsible for designing, building, and optimizing robust data pipelines to ingest, process, and transform data from a variety of databases and external sources. My work supported Data Science teams conducting in-depth research across multiple business verticals, ensuring the seamless integration of large-scale data into the platform to generate actionable insights. I collaborated closely with stakeholders to understand their analytical requirements, ensuring that the data architecture was scalable, efficient, and aligned with business needs. • Developed and maintained ETL pipelines to extract, transform, and load (ETL) data from various sources into Snowflake, ensuring high-quality, timely data for credit risk analysis.• Leveraged GCP services such as Google Cloud Storage for staging data and Cloud Dataflow for batch and stream processing, enabling efficient data ingestion and transformation workflows.• Built credit risk analytics pipelines using BigQuery, Dataproc, and Cloud Storage to support machine learning workflows.• Implemented real-time streaming pipelines using Cloud Pub/Sub, reducing data processing time upto 25%.• Implemented real-time event-driven pipelines using Cloud Pub/Sub, reducing data latency.• Designed and developed data pipelines using Snowflake and Python for credit risk analysis, ensuring high-quality data for decision-making.• Built credit risk analytics pipelines using BigQuery, Dataproc, and Cloud Storage to support machine learning workflows.• Automated workflows with Airflow DAGs and Cloud Composer, improving data orchestration efficiency by 30%.• Developed real-time streaming pipelines with Pub/Sub, reducing data latency for analytics use cases.

Eden Prairie, Minnesota, United States

• Designed and built scalable, distributed data pipelines to ingest and process customer data from multiple sources.• Developed and managed data solutions within a Hadoop Cluster environment using Hortonworks distribution.• Created persistent customer keys to unify customer data across various accounts and systems.• Developed distributed data processing solutions in PySpark to handle large-scale datasets in AWS EMR.

Tampa, Florida, United States

As a Data Engineer at Verizon, I was responsible for designing and implementing scalable data pipelines using GCP tools to support large-scale data analytics. Leveraging BigQuery for data storage and analysis, I enabled the processing of high-volume, real-time data to generate valuable insights for business decision-making. I developed ETL processes in Dataflow, which automated data ingestion, transformation, and loading into cloud data warehouses. Additionally, I used Cloud Storage for raw data storage and Pub/Sub for event-driven data streaming, ensuring high availability and low-latency data flow. Collaborating with cross-functional teams, I optimized pipeline efficiency and ensured data integrity and security.•Integrated data from various sources, including HDFS and HBase, into Spark RDDs to enable distributed data processing and analysis.•Integrated disparate data sources (HDFS, HBase) into a unified platform using GCP services such as Cloud Storage and Data proc.•Built and optimized data pipelines using PySpark for processing and validating large datasets, ensuring data accuracy and quality.•Automated workflows using Apache Airflow and implemented Pub/Sub for real-time data synchronization.•Created data models and Tableau dashboards to visualize operational insights for stakeholders.•Automated workflow orchestration with Cloud Composer (Airflow), improving task efficiency.•Managed real-time streaming using Cloud Pub/Sub to synchronize data between processors.•Built and optimized ETL pipelines using Snowflake and Python for an e-commerce platform, ensuring timely and accurate data processing.•Developed Snowflake data warehousing solutions, improving data availability and retrieval times by 35%.•Developed ETL processes in Data proc to process and clean data for analytics and reporting.•Created Tableau dashboards for end-users, providing visual insights and performance metrics.

Bengaluru, Karnataka, India

I worked as a Data Engineer on a Customer Data Platform project that centralized customer data from various sources, including CRM systems and web analytics platforms. I developed robust ETL pipelines using Apache Spark and PY Spark for data ingestion, transformation, and validation. The project utilized AWS S3 for data storage and Snowflake as the data warehouse, enabling efficient querying and reporting. I collaborated with data scientists to support advanced analytics and machine learning initiatives. Additionally, I created interactive dashboards in Power BI to visualize key customer metrics for stakeholders.• Developed various spark applications using Scala to perform various enrichment of these click stream data merged with user profile data.• Developed Spark code in Python and Spark SQL environment for faster testing and processing of data and loading the data into Spark RDD and doing In-memory computation to generate the output response with less memory usage.• Worked on Scala code base related to Apache Spark performing the Actions, Transformations on RDDs, Data Frames and Datasets using Spark SQL and Spark Streaming Contexts.

Chennai, Tamil Nadu, India

I worked on a project focused on developing a big data processing framework to analyze large volumes of unstructured data. The project involved designing and implementing data ingestion pipelines using Apache Flume and Sqoop to extract data from various sources, including relational databases and log files. I utilized Hadoop MapReduce for data processing and transformation tasks, ensuring efficient handling of large datasets. Additionally, I collaborated with data scientists to integrate machine learning models into the workflow for predictive analytics. Finally, I helped optimize the performance of the Hadoop cluster for improved data processing speeds and resource utilization.• Developed and maintained large-scale data ingestion pipelines using Apache Hadoop components such as HDFS, MapReduce, and Hive to process terabytes of data daily.• Integrated data from multiple sources like relational databases, flat files, and logs into Hadoop HDFS for efficient storage and processing.• Implemented MapReduce jobs to transform and aggregate data, performing batch processing on massive datasets.• Worked with Hive and Pig for querying and analyzing large datasets stored in HDFS, optimizing performance for data analysis.