Tech Lead Data Engineer
CurrentTech Lead Data Engineer | GenAI & LLM ImplementationAWS Bedrock LLM Integration: Developed a system with AWS Bedrock and Flask to auto-tag user profiles with predefined attributes, utilizing S3 Storage, Claudi Sonnet LLM, Cohere LLM, Langchain and Docker.Weaviate Vector Database & Executive Search: Implemented profile management and similarity search using Weaviate vector DB and AWS Bedrock Cohere embeddings, optimizing document handling and query functionality.Cost-Effective Tagging with Open-Source LLMs: Leveraged Llama 3/3.1 and Chroma vector DB for affordable profile tagging, enhancing cost efficiency and profile attribute handling.ETL-Based Standardization: Standardized of identity location details in PostgreSQL using Levenshtein distance for improved data consistency.Identity Resolution ETL: Developed a solution for CRM data accuracy by resolving duplicates and inconsistencies using Levenshtein distance and Soundex algorithms in SQL Server.
