Sai Deep Email & Phone Number

United States

● Created Function as a service is a category of cloud computing services that provides a platform allowing customers to develop, run, and manage application functionalities without the complexity of building and maintaining the infrastructure typically associated with developing.● Implemented a 'serverless' architecture using API Gateway, Lambda, and Dynamo DB and deployed AWS Lambda code from Amazon S3 buckets. ● Developed and maintained applications using Java, JavaScript, and Python, creating efficient data processing solutions and backend services for enterprise-scale logistics operations at UPS.● Implemented CI/CD pipelines using AWS Code Pipeline and Code Build for automated deployment of AWS Lambda functions.● Experience integrating AWS services (e.g., S3, Lambda, DynamoDB) using the AWS SDK for JavaScript, enabling seamless cloud interactions within JavaScript applications.● Partnered closely with Engineering stakeholders to gather requirements, understand project objectives, and design scalable data pipelines needs.● Designed and implemented scalable solutions to extract data from REST APIs using Python, automating daily data pulls that support real-time analytics.● Designed and implemented containerized applications using Docker, improving deployment efficiency and consistency across development and production environments.● Demonstrated expertise in deploying, managing, and orchestrating containerized applications using Kubernetes, ensuring high availability, scalability, and fault tolerance.● Optimized application performance by fine-tuning AWS Lambda function configurations, including memory allocation and execution time.● Integrated AWS Lambda with AWS API Gateway to create scalable and efficient RESTful APIs.

Dallas, Texas, United States

● Extensive working experience in working Microsoft Azure and Hadoop Cloudera Distribution platform.● Designed and implemented ETL pipelines using AWS Glue to extract, transform, and load data from various sources into data lakes and warehouses.● Implemented a 'serverless' architecture using API Gateway, Lambda, and Dynamo DB and deployed AWS Lambda code from Amazon S3 buckets.● Developed Airflow DAGs to schedule and automate end-to-end data pipelines, enabling the seamless execution of daily batch processing tasks and improving overall data pipeline efficiency● Implemented best practices for managing DynamoDB tables, including partition management, data retention policies, and backup and restore strategies.● Integrated Kinesis with various AWS services (e.g., S3, Lambda) to create a seamless data pipeline for ingesting and storing streaming data, enhancing overall data architecture.● Designed and implemented a comprehensive data ingestion pipeline using Apache NiFi to collect and integrate data from various higher education institutions across Texas, including student enrollment, academic performance, and graduation rates.● Implemented data quality checks in Collibra, resulting in a 30% reduction in data discrepancies across multiple educational institutions.● Built efficient ETL/ELT pipelines using tools like AWS Glue, Apache Spark, and Talend to load and transform data from various sources into Amazon Redshift.● Developed data processing scripts and applications using Python and Scala within DLT.● Developed serverless applications using AWS Lambda and API Gateway to automate order processing, reducing manual intervention, and improving operational efficiency

Arlington, Texas, United States

● Orchestrated the deployment of containerized data processing workflows using Google Kubernetes Engine (GKE), improving resource utilization and scalability.● Developed and optimized complex SQL queries in BigQuery, reducing query execution times and improving overall performance.● Experience with converting Teradata schemas, tables, and views to equivalent structures in GCP, particularly BigQuery.● Designed and implemented hybrid database solutions combining NoSQL and SQL databases to leverage the strengths of both technologies for specific application needs.● Built pipelines to transfer and transform large datasets stored in Google Cloud Storage into Snowflake, leveraging Snowflake’s native connectors.● Utilized Talend to integrate diverse datasets for Texas Health Care, including clinical and operational data, ensuring seamless data flow between disparate systems and the data warehouse.● Implemented partitioning and clustering strategies in BigQuery tables, improving query efficiency and reducing storage costs.● Managed the deployment of dbt models to production environments in GCP, using automated deployment tools and monitoring dashboards to ensure stable and reliable data transformations.● Automated data migration workflows using Cloud Composer, orchestrating complex data pipelines to handle diverse data sources and destinations within GCP.

California, United States

● Involved in developing batch and stream processing applications that require functional pipelining using Spark APIs.● Developed Databricks notebooks for data preparation i.e. data cleansing, data validation, et cetera and applied transformations as per our requirement● Involved in building a data pipeline and performing analytics using AWS stack (EMR, EC2, S3, RDS, Lambda, SQS, Redshift).● Developed Sqoop jobs for data ingestion, incremental data loads from RDBMS to HDFS.● Utilised Spark’s in memory capabilities to handle large datasets on S3 Data lake. Loaded data into S3 buckets, then filtered and loaded into Hive external tables..● Involved in developing a custom-built Rest API to support real time customer analytics for data scientists and applications.● Involved in extracting and enriching multiple Cassandra tables using joins in Spark SQL. Also converted Hive queries into Spark transformations.● Developed various data loading strategies and performed various transformations for analysing the datasets by using Hortonworks Distribution for Hadoop ecosystem.● Working knowledge in creation and modification of SQL stored procedures, functions, views, indexes, and triggers● Fetched live data from Oracle database using Spark Streaming and Kinesis using the feed from API Gateway RESTservice.● Automated the process of transforming and ingesting terabytes of monthly data using Kinesis, S3, and Oozie.● Performed ETL operations using Python, Spark SQL, S3 and Redshift on terabytes of data to obtain customer insights.● Developed Oozie workflows for scheduling and orchestrating the ETL process.● Experience with Apache big data Hadoop components like HDFS, MapReduce, YARN, Hive, HBase, Sqoop, Pig● Used Ansible to provision the environment and deployed applications in a CI/CD process using a Jenkins pipeline● Analyzed and optimised pertinent data stored in Snowflake using PySpark and Spark SQL

Hyderabad, Telangana, India

● Involved in import and export of data between Hadoop Data Lake and Relational Systems like MySQL, Oracle usingSqoop.● Working knowledge on creating Kafka topics, partitions and writing custom partitioners classes.● Experienced in writing Spark applications in Python (Pyspark).● Made use of the AWS environment to launch the applications in different regions and implemented CloudFront with AWS Lambda to reduce latency.● Worked and acquired a lot of knowledge in Amazon Web Services (AWS) cloud services like EC2, S3, EMR, EBS, RDS and VPC.● Extracting real time data using Kafka and Spark streaming by Creating D streams and converting them RDD, processing it and storing it into Cassandra.● Configured, deployed and maintained multi-node Dev and Test Kafka Clusters.● Processed and transferred the data from Kafka into HDFS through Spark Streaming APIs.● Experience in building Real-time Data Pipelines with Kafka Connect and Spark Streaming.● Building the Cassandra nodes using AWS & setting up the Cassandra cluster using Ansible automation tools.● Developed Oozie Bundles to Schedule Pig, Sqoop and Hive jobs to create data pipelines.● Developed Hive queries to do analysis of the data and to generate the end reports to be used by business users.● Developed Scala scripts, UDF’s using both Data frames/SQL, RDD/MapReduce in Spark for Data Aggregation, queries and writing data back into RDBMS through Sqoop.● Written storm topology to emit data into Cassandra DB.● Involved in the process of Cassandra data modelling and building efficient data structures