Sai S Datar Email & Phone Number

Basking Ridge, Nj, Us

• Responsible for the execution of big data analytics, predictive analytics and machine learning initiatives.• Implemented a proof of concept deploying this product in AWS S3 bucket and Snowflake.• Utilize AWS services with focus on big data architect /analytics / enterprise Data warehouse and business intelligence solutions to ensure optimal architecture, scalability, flexibility, availability, performance, and to provide meaningful and valuable information for better decision-making.• Developed Scala scripts, UDF's using both data frames/SQL and RDD in Spark for data aggregation, queries and writing back into S3 bucket.• Experience in data cleansing and data mining.• Wrote, compiled, and executed programs as necessary using Apache Spark in Scala to perform ETL jobs with ingested data.• Used Spark Streaming to divide streaming data into batches as an input to Spark engine for batch processing.• Wrote Spark applications for data validation, cleansing, transformation, and custom aggregation and used Spark engine, Spark SQL for data analysis and provided to the data scientists for further analysis.

Petaluma, California, Us

• Designing and building multi-terabyte, full end-to-end Data Warehouse infrastructure from the ground up on Confidential Redshift for large scale data handling Millions of records every day• Worked on Big data on AWS cloud services i.e. EC2, S3, EMR and DynamoDB• Managed security groups on AWS, focusing on high-availability, fault-tolerance, and auto scaling using Terraform templates. Along with Continuous Integration and Continuous Deployment with AWS Lambda and AWS code pipeline.• Implementing and Managing ETL solutions and automating operational processes.• Optimizing and tuning the Redshift environment, enabling queries to perform up to 100x faster for Tableau and SAS Visual Analytics• Wrote various data normalization jobs for new data ingested into Redshift• Advanced knowledge on Confidential Redshift and MPP database concepts.• Migrated on premise database structure to Confidential Redshift data warehouse• Was responsible for ETL and data validation using SQL Server Integration Services.

Columbia, Sc, Us

• Contributed in building our cloud infrastructure in Azure. Automated Cloud deployments using Chef, Python and Azure Cloud Formation templates.• Used kafka stream processing to collect data from producers, configure Topics in Kafka cluster and made the data available for consumers.• Involved in file movements between HDFS and Azure Blob storage and extensively worked with Blob Containers in Azure. • Maintained the repository management tools like Artifactory and Nexus to store the WAR, JAR files which are deployed by using Chef in Jenkins tool.• Created data partitions on large data sets in Blob and DDL on partitioned data.• Implemented rapid-provisioning and life-cycle management for using Azure Virtual Machine and custom Bash scripts.

• Responsible for building the datalake in Amazon AWS, ingesting structured shipment and master data from Azure ServiceBus using the AWS APIGateway, Lambda, and Kinesis Firehose into s3 buckets.• Implemented Data pipelines for big data processing using Spark transformations and Python API and clusters in AWS• Create complex Sql queries in Teradata Data Warehouse environment to test the data flow across all the stages• Integrated data sources from Kafka (Producer and Consumer API) for data stream-processing in Spark using AWS Network