Software Engineer 2
Current* Web Scraping Optimization: Developed and optimized web scrapers using Python (BeautifulSoup, Scrapy) and OpenAI API, reducing manual job vacancy collection by 40 hours weekly. Stored data in AWS S3 for lead generation.* Real-time Monitoring Automation: Automated session monitoring using Puppeteer, Node.js, and AWS Lambda, resolving live session sync issues from DynamoDB, significantly improving system reliability.* OTUS Reporting Enhancements: Designed and implemented Redis and Elasticsearch-based alerting for data discrepancies in OTUS reports, improving troubleshooting and monitoring efficiency.* Complex MySQL Procedure Redesign: Fixed critical OTUS allocation bugs by redesigning complex MySQL stored procedures and integrating Polly for reliable SQL connections, ensuring robust report generation.* Report Management: Managed and delivered OTUS reports using C# .NET Framework with automated cron jobs, enhancing accuracy, timeliness, and error handling in report generation.