Designed and executed complex data processing solutions, optimizing Hadoop jobs for in-depth analysis and developing optimal data pipeline architecture. Built efficient data pipelines using PySpark to generate optimized data sets in formats like Avro and Parquet. Ensured over 95% data quality through data analysis and profiling, supporting business decision-making and enhancing team efficiency. Skilled in data visualization tools such as Tableau and Power BI, effectively conveying insights through compelling dashboards and reports for business intelligence.