Software Engineer
CurrentRoles & Responsibilities:> Proficient in Data Ingestion utilizing Scrapy, Selenium, and Requests for efficient and comprehensive data collection.> Contributed significantly to code base development, focusing on clean, readable, and standardized code following PEP8 guidelines.> Creating and managing item and validation pipelines to maintain data integrity and quality.> Experience in developing and managing request middleware to optimize web scraping processes.> Collaborating and training fresh graduates, contributing to team growth and development.> Specialized in the development of crawlers for data collection, focusing on maintenance, bug fixing, and implementing proxy usage for bypassing blocks.> Conducted testing of crawlers and performed code reviews to maintain high-quality standards and best practices.