Currently Working on-Data and AI Platform:Spearheading the development of a cutting-edge Data and AI Platform designed to empower organizations with actionable insights and advanced analytics capabilities. Leveraging state-of-the-art technologies to deliver scalable, secure, and high-performance data platform.Consulting: Data and AI In addition to leading platform development, I offer consulting services for multiple customers across various industries, specializing in Data… Show more Currently Working on-Data and AI Platform:Spearheading the development of a cutting-edge Data and AI Platform designed to empower organizations with actionable insights and advanced analytics capabilities. Leveraging state-of-the-art technologies to deliver scalable, secure, and high-performance data platform.Consulting: Data and AI In addition to leading platform development, I offer consulting services for multiple customers across various industries, specializing in Data Engineering, Cloud Architecture, and Generative AI (GenAI) solutions. My consulting expertise includes:Designing robust cloud infrastructures to optimize scalability, performance, and cost-efficiency.Architecting and implementing data pipelines, enabling seamless data integration and transformation for real-time analytics.Delivering custom GenAI solutions that drive business insights and automation, leveraging the latest advancements in large language models and AI frameworks.Providing strategic guidance on AI adoption, helping organizations unlock the power of machine learning and predictive analytics to stay ahead of the competition.With a deep focus on business outcomes, my consulting approach ensures organizations can navigate the complexities of modern data ecosystems while accelerating innovation and driving measurable impact.Tools and Technologies : Platform: DataBricks , PalantirCloud: Azure, AWS, GCPData Engineering tools: ADF, Synapse Analytics, DataBricks, Unity Catalogue, SQL, PySpark, Hive, T-SQL, PL/SQL,WebServices: REST, SOAP, gRPCProgramming Language: Python, Java, Rust Show less

Technical Delivery and Architecture:Accountable for technical delivery across solution, cloud, and data projects.Leveraged enterprise architectural standards and patterns to create solutions that deliver essential business capabilities for US and LATAM.Collaborated with development managers, IT & business directors, and VPs to gain consensus and drive project decisions.Led the Discovery and Design phases of multiple data projects, engaging stakeholders to define project… Show more Technical Delivery and Architecture:Accountable for technical delivery across solution, cloud, and data projects.Leveraged enterprise architectural standards and patterns to create solutions that deliver essential business capabilities for US and LATAM.Collaborated with development managers, IT & business directors, and VPs to gain consensus and drive project decisions.Led the Discovery and Design phases of multiple data projects, engaging stakeholders to define project scopes.Data Architecture and ETL:Developed and fine-tuned PySpark code and machine learning models accelerating data processing and improving accuracy.Created and managed indexes while closely monitoring execution plans to optimize query performance and enhance system efficiency.Extracted and ingested data from diverse sources into a centralized Data Lake Using Streaming, Flat Files, Kafka, DB and many other sources, Implemented robust ETL processes with PySpark , SQL over Delta Tables and Delta Live Tables and ensuring data quality and consistency.Data Migration: Perform data migration from legacy systems (AS400, HDFS) to cloud and Data Lakehouse based architecture , getting performance boost up to ~31%+ in performance.Cloud Optimization and Automation:Monitored cloud performance to optimize resource utilization, reduce costs, and implemented incremental data loading strategies for efficiency.Architected and designed secure, efficient, and resilient solutions for Azure and GCP, compliant with customer cloud standards.DevOps: Implemented DevSecOps processes and CI/CD pipelines using tools like Azure DevOps , Jira, Ansible, Jenkins.Designed strategies and tools to deploy, monitor, and administer cloud applications and underlying services for Azure and GCP.Led containerized web application development, enhancing scalability and maintainability of UI components.Optimized legacy Hive and T-SQL queries, significantly improving performance and reducing execution times. Show less

Domain: Re-Insurance.Contributing as senior engineer. -------- Azure Cloud and DataBricks------Performed data transformations before pushing from raw to clean stages.Worked with Slowly Changing Dimensions (SCD) Type 2 data.Built multi-source data ingestion pipelines into a Lakehouse, followed by cleaning and transforming data for business use and machine learning models.Worked extensively on Databricks Notebooks within Azure… Show more Domain: Re-Insurance.Contributing as senior engineer. -------- Azure Cloud and DataBricks------Performed data transformations before pushing from raw to clean stages.Worked with Slowly Changing Dimensions (SCD) Type 2 data.Built multi-source data ingestion pipelines into a Lakehouse, followed by cleaning and transforming data for business use and machine learning models.Worked extensively on Databricks Notebooks within Azure Cloud. --------Palantir Cloud Foundry-----------Created syncs from source systems using JDBC connectors, ingesting data from raw to clean stages.Used Workshop to create business reports aimed at understanding and mitigating risk, and performing assessments on processes.Utilized Quiver, a visualization tool within Palantir Foundry, to create charts, graphs, and maps for visualization-based reports.Migrated older reports from Contour and other visualization tools to Quiver and Workshop-based reports.Integrated Palantir Foundry with SharePoint for data ingestion from DataLake, REST, SharePoint and JDBC based services.Environment Setup:Created Ansible Scripts to run on Azure cloud to install Big Data Components,Setup dockers and used Ansible on Virtual Machines.Tech Stack: - Palantir Foundry (Quiver, Workshop, Contour, Build Schedule, Data Sync, Data Lineage, Code Workbook, Repos), PySpark, AWS, Spark, Python, SQL.Methodologies: - Agile Show less

Building customer recommendation system for Retail Customer.- Led team of 5 engineers.- Re-written old sklearn, pandas, python code into SparkML. - troubleshooted issues over GCP environment.- Consumed RESTful APIs in Python to extract information from Database.Technologies :- Spark, PySpark, Python, GCP(Google Cloud, DataProc, Cloud Function, Cloud Composer, Cloud DataFlow, GCS, BigQuery).Methodologies: Agile.

Involved in End to End Implementation of the project(Java, WebServices and Spark Implementation).Work closely with BU and business Analyst to finalize the requirement.Written new data pipeline using python and spark based API i.e. pyspark.Created the Rest API extensively using Spring Boot.Extensively worked on Ad-hoc requirements.Consume the Java based Rest API in PythonIntegration of the python scripts with Java APIs.Written Extensively REST APIs using Java and… Show more Involved in End to End Implementation of the project(Java, WebServices and Spark Implementation).Work closely with BU and business Analyst to finalize the requirement.Written new data pipeline using python and spark based API i.e. pyspark.Created the Rest API extensively using Spring Boot.Extensively worked on Ad-hoc requirements.Consume the Java based Rest API in PythonIntegration of the python scripts with Java APIs.Written Extensively REST APIs using Java and Spring framework to create the matching project which will be execute later on the Spark Cluster.Fixing functional and performance defects in the production system.Tech Stack:Framework: Spring Boot, Spring Data, Spring JPA, Apache Spark. Tools used in ETL : Hive, SparkSQL, PySpark.Programming Language: Java8, Python, SQL. Data Storage Mechanism : Hbase, PostgreSQLData Munging: Python Data Storage Mechanism : HDFS, HbasePlatform: Palantir Foundry Similar to Cloudera or Hortonworks.Methodology: Agile Show less

Integrated other banking products into Spectrum to extend its functionality as per the requirements given by the BU.Added analytics using PySpark as per the requirements given by the BU to improve the business outcome of the Credit Unions and Banks.Responsible to design and enable existing UI into Google Chrome and Microsoft Edge using promise and Java Script.Responsible to re-write old C++ and Core Java Logic into the Latest Java 8 version.Fixing functional and performance… Show more Integrated other banking products into Spectrum to extend its functionality as per the requirements given by the BU.Added analytics using PySpark as per the requirements given by the BU to improve the business outcome of the Credit Unions and Banks.Responsible to design and enable existing UI into Google Chrome and Microsoft Edge using promise and Java Script.Responsible to re-write old C++ and Core Java Logic into the Latest Java 8 version.Fixing functional and performance defects in the production system.Platform & Skills: Core Java, J2EE, JSP, Servlets, Spring, PySpark, XML, Web services, JDBC, Oracle 10g, Linux, JDK 1.8, Glassfish, Tomcat, log4j, Jersey, Ant, Maven Show less

Used Technologies:-Java:- Core Java, Servlet, JSP, Spring, Maven, Jboss, Tomcat