Working in Big Data Platform for Cognizant for past 3+ years that includesdevelopment and testing of data pipelines and loading of data throughETL jobs.· Designed, developed, and maintained data pipelines to ingest, process,and transform large volumes of structured and unstructured data from various sources into usable formats for analytics and reporting purposes.· Worked closely with data scientists, analysts, and stakeholders to understand business requirements and translate them into technical specifications for data solutions.· Experienced in delivering data & analytics solutions using Agile deliverymethodology.· Developed Scala applications for data processing and analysis usingApache Spark framework.· Implemented efficient ETL processes using Apache Spark to cleanse,transform, and load data into data warehouses and data lakes.· Ingested various files of different formats like CSV, Parquet, JSON etc fromvarious sources like SFTP and DB2 leveraging QLIK ingestion. Also workedon ingesting streaming data from streaming source using Spark Streaming.· Utilized Databricks platform for data engineering tasks such as data ingestion, transformation and modelling.· Optimized the performance of the Spark Jobs in Databricks up to 50 percent more efficient than the existing jobs by implementing photon acceleration and suitable spark node type.· Created and optimized SparkSQL queries for efficient data retrieval andmanipulation. Also very much proficient in SQL.· Leveraged cloud platforms such as AWS to build scalable and cost effective data solutions, utilizing services like S3, Athena, Glue and Redshift.· Automated data workflows and scheduling using Talend/ Stonebranch/AirFlow, improving data pipeline reliability and reducing manual intervention.· Leveraged Denodo to expose curated tables to business users, enablingaccess to the business users for their critical data insights.· Experienced in working in a fully CI/CD managed environment tools suchas git, Jenkins.· Optimized query performance and data storage efficiency by tuningdatabase configurations and implementing indexing strategies.· Conducted data quality assessments and implemented data governancebest practices to ensure data quality from 95 percent to 100 percentthreshold for consistency, quality and security.· Collaborated with cross-cultural teams to troubleshoot issues, identifyroot causes and implement solutions in a timely manner.· Experienced in Agile ways of working in SCRUM setups and Atlassian toolslike JIRA and Confluence