Sai Chandra Email & Phone Number

United States

United States

• Experience in building and architecting multiple Data pipelines, end to end ETL and ELT process for Data ingestion and transformation in GCP and coordinate task among the team. • The roles include creating data pipelines from application databases to Data Lake and warehouse and create stream and batch data processing pipelines. • Build data pipelines in airflow in GCP for ETL related jobs using different airflow operators. • Created python scripts to ingest data from on-premise to GCS and built data pipelines using Apache Beam and Data Flow for data transformation from GCS to Big query • Maintaining more than 16 PB, 300 nodes Cloudera's distribution Hadoop production and dev clusters. Perform daily health checks, work on alerts, and other related tasks. • Created Hive databases, tables and queries for data analytics as and when needed, integrate them with data processing jobs, and drop them as a part of cleanup. Also working on maintaining HBase database used by other applications. • Leveraged Google Cloud Platform Services to process and manage the data from streaming and file-based sources. • Designed and developed in house database visualization tool built on Oracle Application Express platform that visualize the real-time database health information as well as shows management reports. • Experience in moving data between GCP and Azure using Azure Data Factory. • Experienced as AWS cloud engineer to create and manage EC2 instances, S3 buckets, and configure RDS instances. Monitoring the performance, spinning off the instances regularly, and help developers to get access to the cloud instances.

Dania, Florida, United States

• Worked on AWS Data pipeline to configure data loads from S3 to into Redshift. • Using AWS Redshift, I Extracted, transformed and loaded data from various heterogeneous data sources and destinations • Created Tables, Stored Procedures, and extracted data using T-SQL for business users whenever required. • Performs data analysis and design, and creates and maintains large, complex logical and physical data models, and metadata repositories using ERWIN and MB MDR • I have written shell script to trigger data Stage jobs. • Assist service developers in finding relevant content in the existing reference models. • Like Access, Excel, CSV, Oracle, flat files using connectors, tasks and transformations provided by AWS Data Pipeline. • Utilized Spark SQL API in PySpark to extract and load data and perform SQL queries. • Worked on developing Pyspark script to encrypting the raw data by using Hashing algorithms concepts on client specified columns. • Responsible for Design, Development, and testing of the database and Developed Stored Procedures, Views, and Triggers • Developed Python-based API (RESTful Web Service) to track revenue and perform revenue analysis. • Compiling and validating data from all departments and Presenting to Director Operation. • KPI calculator Sheet and maintain that sheet within SharePoint.

New York, United States

• Designed and Implemented Sharding and Indexing Strategies for MongoDB servers. • Optimizing pig scripts, user interface analysis, performance tuning and analysis. • Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting on the dashboard. • Implemented the import and export of data using XML and SSIS. • Involved in Planning, Defining and Designing data base using ER Studio on business requirement and provided documentation. • Responsible for migration of application running on premise onto Azure cloud. • Used SSIS to build automated multi-dimensional cubes. • Wrote indexing and data distribution strategies optimized for sub-second query response • Developed a statistical model using artificial neural networks for ranking the students to better assist the admission process. • Wrote scripts and indexing strategy for a migration to Confidential Redshift from SQL Server and MySQL databases. • Implement software enhancements to port legacy software systems to Spark and Hadoop ecosystems on Azure Cloud.

Hyderabad, Telangana, India

• Deployed Lambda and other dependencies into AWS to automate EMR Spin for Data Lake jobs • Scheduled spark applications/Steps in AWS EMR cluster. • Extensively used event-driven and scheduled AWS Lambda functions to trigger various AWS resources. • Implemented advanced procedures like text analytics and processing using teh in-memory computing capabilities like Apache Spark written in Scala. • Developed spark applications for performing large scale transformations and denormalization of relational datasets. • Developed and executed a migration strategy to move Data Warehouse from SAP to AWS Redshift. • Loaded data into teh cluster from dynamically generated files using Flume and from relational database management systems using Sqoop. • Used Spark Streaming to divide streaming data into batches as an input to spark engine for batch processing. • Worked on analyzing Hadoop cluster and different Big Data analytic tools including Pig, hive, HBase, Spark and Sqoop. • Exported data from HDFS to RDBMS via Sqoop for Business Intelligence, visualization, and user report generation. • Loading teh data from multiple Data sources like (SQL, DB2, and Oracle) into HDFS using Sqoop and load into Hive tables. • Performed Real time event processing of data from multiple servers in teh organization using Apache Storm by integrating wif apache Kafka. • Performed Impact Analysis of teh changes done to teh existing mappings and provided teh feedback • Involved in complete Implementation lifecycle, specialized in writing custom MapReduce, and Hive • Extensively used Hive/HQL or Hive queries to query or search for a string in Hive tables in HDFS

Hyderabad, Telangana, India

Involved in implementation of the project went through several phases namely: data set analysis, preprocessing data set, user-generated data extraction, and modeling.Participated in Data Acquisition with the Data Engineer team to extract historical and real-time data by using Sqoop, Pig, Flume, Hive, MapReduce, Land HDES.Wrote user-defined functions (UDFs) in Hive to manipulate strings, dates, and other data.Performed Data Cleaning, features scaling, features engineering using pandas, and NumPy packages in python .Process Improvement: Analyzed error data of recurrent programs using Python and devised a new process to reduce the turnaround time of the problem's solutions by 60%Worked on production data fixes by creating and testing SQL scripts.Deep dived into complex data sets to analyze trends using Linear Regression, Logistic Regression, Decision Trees.Prepared reports using SQL and Excel to track the performance of websites and apps.Performed Data Collection, Data Cleaning, Data Visualization, and Feature Engineering using Python libraries such as Pandas, Numpy, MatPlotLib, and seaborn.