Director Of Data Science And Ai
Current- Designing a custom RAG-based question-answering system using VannaAI and GPT-4o-mini to enable clients to query platform data seamlessly. It validates SQL-compatible questions, retrieves relevant question-SQL pairs, schema, and metadata, and uses LLMs to generate SQL queries based on user access levels. Query outputs are transformed into synthetic data and visualized with LLM-generated Plotly code, delivering interactive insights directly on the platform.- Developed an AI-powered event detection system using GPT-4o-mini with retrieval-augmented generation (RAG), automating alert classification and reducing manual review by 200+ hours per client monthly. It reuses historical event insights from Pinecone’s vector store, preventing redundant analysis, and streamlines SME validation, boosting operational efficiency.- Built a fully automated AutoML and MLOps framework, automating EDA, feature engineering, model selection, deployment, and monitoring via the platform frontend. Integrated Dask, Ray, and MLFlow, reducing model preparation time by 95% (from one week to 2 hours), cutting manual workload by 90%, and enabling continuous optimization with automated drift detection, re-training, and monitoring.- Re-engineered ML model deployment by transitioning from single-model Kubernetes pods to a Celery-based multi-model system, cutting hosting costs by 75% ($4 to $1 per model/month) and reducing EC2 instances by 80% (from 30 to 5-6). Increased resource allocation per model 10x and ensured seamless library upgrades with backward compatibility.- Revamped data orchestration, migrating from Airflow to Dagster with Celery-based parallel processing, increasing pipeline speed by 50% and reducing infrastructure costs by 70%. Consolidated ETL, physics calculations, and ML workflows, reducing I/O overhead via Redis and enabling scalable parallel execution.- Built and led a high-performing 10-member data science team, maximizing productivity through strategic delegation.
