Principal Consultant
CurrentRETAIL SOLUTIONS PROVIDER OF DATA-DRIVEN CUSTOMER INSIGHTS Gen AI experience using langchain to summarize Sharepoint documents into RAG vector databaseEnhanced Data and Analytics Platform's capabilities in Databricks by migrating from parquet to Unity Catalog, handling terabytes of data and billions of transaction records.Modernized Databricks ETL pipelines to improve efficiency and scalability.Guided the team in adopting best practices, including formalized design reviews, PR standardization, and team training sessions.Focused on building team competencies in Python programming, command-line tools, and SDK usage.FORTUNE 500 FINANCIAL SERVICES COMPANYManaged privacy-related ingestion and processing, ensuring compliance and ethical handling of sensitive customer data utilizing Spark and Python technologies.Processed opt-outs, managed over 200 data sets for proper structure and organization.Collaborated with engineers and analysts to ensure accessibility and usability of data.Provided production support, monitored systems, and responded to alerts.Deployed new features in support to data privacy initiates of the organization.CONSULTANT FOR HIGHLY RANKED FINANCIAL SERVICES PROVIDER Delivered significant analytics capabilities across multiple lines of business.Implemented a scheduled ETL ingestion from Microsoft SQL Server and other ad-hoc data sources using JDBC into AWS Glue/Lake Formation utilizing AWS Lake Formation, Glue, PySpark, and other AWS services.Reverse-engineered SQL Server stored procedures as a basis for the PySpark code, including implementing multiple ingestion methods.Ensured compliance with regulatory, legal, and information security requirements.