Senior Software Engineer
Bengaluru, Karnataka, India
Research, Discovery, and Funding platform for Indian Companies and Investors, featuring an extensive database with detailed profiles of over 150k Companies, insights on over 4.5k Funds, 29k HNIs, and more.- Engineered a company-wide Change Data Capture (CDC) system utilising MongoDB, Async Python, and RabbitMQ to generate & consume MongoDB collection data changes, to facilitate real-time data synchronisation and event-driven integration between multiple applications.- Led the migration of the EPFO project from Django to Scrapy, creating a fault-tolerant, self-recovering scrapy spider that ensures complete data integrity with every scrape and achieves scrapes about 50% faster than the previous implementation. Integrated technologies that include Scrapy, FastAPI, Celery, RabbitMQ, and Pydantic.- Designed and implemented a centralised Lists & Filters management screen using Python, Elasticsearch, and Django ORM, which promotes user engagement, profile interaction, and provides enhanced data change updates and tracking features.- Developed and deployed a Structured Logging tool, now integrated across over 20 projects, enhancing application monitoring and observability, also significantly reducing testing and debugging time.- Utilised GitHub Actions for Kubernetes deployment and GitLab CI/CD for Python package distribution. Monitored Kubernetes deployments and app stability using kubectl.