As an Azure Data Engineer at R Systems International Pvt Ltd, I specialized in building and optimizing data solutions to ensure the integrity and usability of data. I designed and implemented scalable ETL pipelines using Azure Data Factory, PySpark, Databricks, and ADLS Gen2, leveraging PySpark for efficient distributed processing and advanced data transformations.A key aspect of my role was data cleaning: I conducted in-depth data exploration to identify and rectify corrupt rows, inconsistencies, and anomalies. This involved applying a variety of data transformation techniques to standardize and cleanse datasets, ensuring that the data was accurate, complete, and ready for further analysis.In addition to data cleaning, I automated data workflows from various sources, integrating secure credential management through Azure Key Vault. I optimized SQL queries and built data warehouses in Azure Synapse to enhance query performance and reporting capabilities.My experience also includes solving over 400 data structures and algorithm challenges, which further refined my analytical and problem-solving skills. My efforts in data engineering consistently supported strategic decision-making and drove data efficiency.