As a Data Solutions Architect, I designed and implemented scalable data management solutions utilizing Azure Synapse, Azure Data Lake, and Azure SQL. I orchestrated complex ETL/ELT processes using Azure Data Factory, enabling efficient data movement and transformation from multiple sources. Leveraging Azure Databricks, I developed PySpark applications for data processing and integration, optimizing data input and ensuring seamless data flow.I built and automated data pipelines, transferring data from ADLS Gen2 to Azure Synapse Analytics with Parquet files for optimal performance and storage. Collaborating closely with cross-functional teams, I ensured data availability and accuracy for business-critical functions. I created and optimized relational and non-relational data models in Azure Synapse, supporting advanced business intelligence and analytics initiatives.I also developed Power BI dashboards to deliver real-time insights, facilitating data-driven decision-making for stakeholders. My work enforced strong data governance through role-based access control and adherence to GDPR and HIPAA standards. Additionally, I optimized PySpark applications for cost-efficiency in the cloud and participated in Agile ceremonies, promoting continuous improvement and collaboration.By implementing CI/CD pipelines with Azure DevOps and Jenkins, I reduced manual deployment effort and improved delivery speed by 30%. I also configured and automated Jenkins build pipelines, driving a culture of automation and enhancing delivery efficiency.

Designed and implemented end-to-end data solutions on Azure, leveraging Azure Data Factory for ETL processes and Azure Databricks for data processing with PySpark. Developed Spark applications for data extraction and transformation, integrated PySpark Streaming for real-time data analysis, and automated batch job management using Control-M. Spearheaded the automation of data query tasks, performance tuning of SQL queries, and created custom PySpark transformations. Collaborated with Agile teams to ensure seamless data pipeline integration. Integrated Power BI with Azure data sources, set up automated refresh schedules, and employed techniques to manage historical data. Managed source code with GitHub and implemented data governance policies, reducing data quality issues by 30%. Experience with CI/CD systems using Git, Jenkins, and Bitbucket, alongside automated testing and continuous monitoring with Azure Monitor.

As a Azure Data Engineert, I have led the development of complex SQL queries for batch data processing, enabling efficient data aggregation, summarization, and reporting. I implemented PySpark scripts that reduced costs by 30% through migrating customer data from SQL Server to Hadoop, and ensured data accuracy and consistency via comprehensive data cleansing and quality checks. My expertise includes transforming and analyzing data across multiple formats (e.g., JSON, CSV, Parquet) using Spark SQL and Python on Azure Data Lake, as well as converting Hive/SQL queries into Spark transformations. I’ve managed end-to-end data solutions, from ingestion to real-time analytics, while also successfully migrating on-premises data solutions to Azure cloud infrastructure. Additionally, I’ve built machine learning pipelines, developed modular code for feature engineering, and maintained dimension/fact tables for data warehouses, all contributing to enhanced decision-making and analytics.

In this role, I collaborated with data analysts and operations teams to collect and analyze data from internal systems, while developing and optimizing software applications that meet industry standards and user requirements. I focused on improving application performance, detecting and resolving software faults, and ensuring the security of software systems. Additionally, I created processes for data integration, migration, and automation, while validating data accuracy across multiple formats. I also provided technical support, maintained comprehensive documentation, and worked to ensure project timelines were met. My work emphasized continuous improvement, staying current with evolving technologies, and ensuring the overall quality of both software and data solutions.

Designed and deployed ETL pipelines for batch data processing, leveraging Informatica to streamline data integration and normalize data structures. Developed complex Teradata SQL queries and used utilities like Mload, Tpump, and FastExport to manage data effectively. Played an active role in Scrum ceremonies, translating client requirements into actionable user stories. Performed impact analysis, debugging, and root cause investigations to resolve data quality issues and enhance system reliability. Created data pipelines, verified data accuracy, and implemented security measures. Collaborated with cross-functional teams, delivered insights through data visualizations, and supported business intelligence projects.