1. Developing data pipelines to onboard and process data for the CAAP platform2. Ensuring data quality and consistency through data cleaning, transformation, and integration processes.3. Work with Product Tech Team and build Robust and scalable data integration (ETL) pipelines.4. Assisted in developing ETL pipelines using Azure Data Factory5. Experience in handling failures in Production Environment w.r.t issue root causing and fixing them.6. Monitor and troubleshoot data pipelines to ensure data availability and resolve issues promptly.7. Developing data models and maintaining data architecture to support data analytics and business intelligence reporting.8. Involved in Migrating Data flows to CTS(CAAP Transformation Standards) using pyspark. Worked on unit testing for the code converted to CTS9. Experience in creating Dataframe, Transformations, Designing, Developing, and maintaining data processing using Pyspark and optimizing data ingestion.10. Write efficient SQL or PySpark queries for data extraction and transformations, create PySpark transformations using the framework provided, and perform unit testing for existing PySpark transformations.11. Documenting data processes and maintaining data dictionaries