Design, develop, and maintain scalable data pipelines using Python, Spark, Kafka, and Apache Flink.Implement ETL processes using Airflow to automate data workflows.Optimize and manage databases, including Greenplum, Impala, MS SQL, and Postgres SQL.Develop and optimize complex SQL queries using PL/SQL, T-SQL, and other SQL variants.Collaborate with data scientists and analysts to understand data requirements and deliver solutions that meet their needs.Monitor and troubleshoot data pipelines and systems using Grafana and other monitoring tools.Implement data governance and ensure data quality and integrity across all systems.Manage version control and collaborate on code using Git.Use containerization and orchestration tools like Docker and Kubernetes to build and manage scalable data solutions.Develop and execute unit tests to ensure the reliability and accuracy of data processing and transformation logic.Stay current with industry trends and best practices in data engineering and related technologies.

Develop ETL/ELT pipelines.Workflow Scheduling with orchestration tool Airflow.Reduced ETL costs by implementing partitioned and clustered tables and SQL optimization.Was responsible for data integration from different sources RDBMS and file sources to DWH.Database Development, ETL / Data Migration Development , T-SQL, PL/SQL .Maintenance, testing and troubleshooting database issues.DWH creation and optimization for reporting purposes.BI Semantic modelling, reporting and dashboarding on BI tools.Database development using T-SQL.Web scrapping using python.Manipulation, cleaning & processing data with Python.

Design and develop database solutions using Microsoft SQL Server (MSSQL) and T-SQL.Write complex SQL queries, stored procedures, and functions to support various applications and business processes.Optimize database performance through indexing, query optimization, and database tuning.Develop and maintain data models and database schemas to support application development and data analysis.Utilize LogiX to design and create data visualizations, reports, and dashboards.Collaborate with cross-functional teams to gather and analyze requirements, ensuring that database solutions align with business needs.Troubleshoot and resolve database issues, including performance problems and data inconsistencies.Develop and execute unit tests to verify database functionality and performance.Document database designs, processes, and best practices.