 Designed and developed robust and scalable data pipelines, leveraging technologies and Large-scale datasets such as Apache Spark, RDBMS, NoSQL, Spring Boot, Cosmos DB, Cassandra, and GRA orchestration to Snowflake, Kafka, Hadoop, Flume, and SQL, to ensure seamless data flow and accessibility for analytical purposes. Implemented Kubernetes-based solutions for resource optimization, resulting in a 25% reduction in infrastructure costs while maintaining high-performance levels. Expert in designing and optimizing ETL processes using SSIS, building multidimensional models with SSAS, and creating insightful reports with SSRS. Implementing new tools and best practices, streamlining workflows, finding bottlenecks, and streamlining the development and deployment process are all ways to continuously assess and enhance the CI/CD pipeline. Experienced in utilizing DevOps and Quality tools like Azure DevOps, GitHub actions, and Jenkins to streamline workflows and ensure code integrity and quality. Writing Scala code to perform data analysis tasks such as data cleansing, aggregation, and transformation. With Snowflake, I've designed and implemented scalable and reliable data pipelines to handle large volumes of data efficiently. I've optimized data systems to improve reliability, efficiency, and quality, collaborating closely with peers, analysts, and product teams. Expertise of creating scalable and effective data processing pipelines, ETL procedures, and streaming apps  Performed end-to-end Architecture and implementation assessments of various AWS services  Implemented data security measures in AWS, including IAM roles and policies, to ensure compliance with industry standards and regulations. knowledgeable about utilizing Git to work with cross-functional teams, settle disputes, and smoothly integrate code updates. On a day-to-day basis, experienced Agile Scrum methodology and Jira ticket system for project development.

 Implemented and managed robust CI/CD pipelines for data processing workflows using Jenkins, ensuring smooth and automated deployment of ETL processes. Experienced data engineer proficient in Java, SSIS, SAAS, and SSRS for seamless ETL processes and insightful reporting. proficient about creating idiomatic, effective, and clean Golang code for developing microservices, APIs, and applications for data processing. Worked with different Big Data components like Spring, Hive, Apache Spark, Tableau, PostgreSQL, SQOOP & HBase, Kafka, and Docker for data processing and storage. Implement a Warehouse Management System (WMS) connection to simplify data flows between logistics and inventory systems. improved data accessibility and accuracy, which led to a 20% increase in the effectiveness of inventory management. Writing unit tests and documentation for Scala code to ensure reliability, maintainability, and ease of understanding for other team members. Create interactive dashboards and visualizations using tools like Tableau, Power BI, or custom-built solutions to present data-driven insights to stakeholders effectively. Leveraged Python's extensive libraries including NumPy, Pandas, and sci-kit-learn to preprocess and analyze large-scale datasets. Proficient in advanced techniques such as Logistic Regression, Generalized Linear Models, and decision trees for accurate predictive modeling. Leveraged cross-functional expertise to bridge the gap between data engineering and data science teams, facilitating effective communication and knowledge sharing.  competent in cloning repositories, committing changes, branching, merging, rebasing, and marking releases using Git commands. Every day, working with the Jira ticket system and Agile Scrum technique for the creation of projects.

 Proficient in developing and optimizing data pipelines, with expertise in scripting languages like Python, Java, SQL, and Informatica and advanced skills in Windows Batch scripting. Implemented efficient ETL processes using IIS DataStage and Informatica, ensuring seamless data integration and transformation. incorporating automated testing frameworks (such as Scala Test and JUnit) into the CI/CD pipeline to verify the accuracy and dependability of data pipelines and applications. Implementing Spark java pipelines to do the ETL processing on top of the raw HDFS data and writing into Hive table. Combining Scala-based apps with databases, data warehouses, and data visualization tools, as well as other elements of the data ecosystem. Implemented real-time data processing solutions using streaming technologies (e.g., Kafka Streams, Spark Streaming) to enable timely data analysis and decision-making. Experienced in Designing and building data processing pipelines using tools and frameworks in the Hadoop ecosystem. Proficient in ETL processes using Azure Data Factory, Kubernetes, Apache Spark SQL, GraphQL, and pySpark for Azure Data Lake Analytics using technologies such as Node.js and Spring Boot.  Maintain version control of data pipeline code and infrastructure configurations using Git, and document data processes, data flows, and system architectures thoroughly. Expertise in building Databricks notebooks for data extraction, cleansing, loading, testing, and validation to Azure SQL DB. Cosmos DB, Cassandra, and CCM from sources like DB2 and Teradata.  Proficient in deploying pipelines in Azure Data Factory using JSON scripts.