Rajal Soni Email & Phone Number

Schererville, IN, US

United States

• Using the Agile methodology to build the different phases of the Software development life cycle.
• Engineered and executed efficient data processing ETL pipelines with AWS services, resulting in a 20% reduction in data ingestion time and a 30% improvement in data availability.
• Wrote AWS Lambda functions in Python for AWS’s Lambda which invokes Python scripts to perform various transformations and analytics on large data sets in EMR clusters.
• Generated and maintained a scalable data infrastructure, harnessing cloud-based technologies such as AWS, leading to a decrease in infrastructure costs and a 50% increase in data processing capacity.
• Designed and implemented bronze, silver, and gold layers of data quality using PySpark and Spark SQL in Databricks, resulting in 15% improvement in overall data quality and 20% fewer data-related errors.
• Implemented Change Data Capture (CDC) mechanism using triggers and event handlers in AWS Lambda and SNS to enable real-time data synchronization and event-driven processing.

United States

• Analyzed SQL scripts and designed the solutions to implement using PySpark.
• Streamlined an OLAP solution for sensor data history, used by all teams of the project for analysis and process improvement.
• Managed a 1 PB data lake, ensuring data availability and reliability for business intelligence and reporting purposes.
• Assisted in the database design and performance optimization using AWS DynamoDB, and AWS Redshift resulting in 25% faster response times.
• Designed and developed the ETL jobs using AWS Glue Studio- extracted data from Workday using APIs, transformed the data using Python and Pyspark scripts, and loading the transformed data into S3 buckets for reporting.
• Involved Map Reduce/Spark Python modules for machine learning and predictive analytics in Hadoop on AWS.

India

• Worked on Agile project execution methodology.
• Involved in the complete Implementation lifecycle, specialized in writing custom MapReduce, and Hive.
• Developed complex queries and designed SSIS packages to extract, transform, and load (ETL) data into data warehouses data marts.
• Queried database in Snowflake data warehouse and utilized SPSS for statistical analysis of panel data
• Implemented dynamic scaling mechanisms in Azure, leading to a 25% decrease in resource utilization during low-traffic periods and ensuring optimal resource allocation during peak loads.
• Implemented time-based windowing strategies in Azure Stream Analytics, optimizing the process of streaming data.