A web scraping service has been developed with Django Rest API that works independently of any site structure (list, span, etc tag, pagination), XPaths, HTML tags or URLs. This service extracts data such as products and companies by using keywords (for example “company”, “companies”, “our companies”) determined according to the data to be scraped. The service scrapes the websites of companies from the site and collects company contact information by going to these websites. A filtering algorithm was also developed to ensure that the scraped information is searched in the right place on the sites and to improve the scraping performance of the service. The management of the scraping data obtained by the users, operations such as sending SMS and e-mails are supported by a backend developed using ASP.NET Core Web API. The user interface was developed using React and TailwindCSS technologies in a modern and responsive manner.Frameworks & Libraries:Django Rest FrameworkDjango QRedisASP.NET Core Web APIEntity Framework CoreMediatRAutoMapperFluentValidationFluent MailTwilio SMSPostgreSQLPostmanDockerDocker ComposeReactTailwindCSSSwagger OpenAPIGoogle RecaptchaiText LibraryMinIORabbitMQPatterns:CQRS PatternMediator PatternClean ArchitectureRepository PatternCross-Cutting Concerns:ValidationPipelineBehavior (FluentValidation)LoggingPipelineBehavior (Serilog, Graylog)CachingPipelineBehavior (Redis)Tools & Utilities:SerilogGraylogPagination ConfigurationSwagger OpenAPIRedis-based IP Rate LimitingSoft Delete (EF Core)Ignore Query Filter (EF Core)Toaster NotificationsStorage & Indexing:PostgreSQLElasticsearchMongoDBAmazon S3-compliant MinIOContainerization:DockerDocker ComposeTesting:PostmanAuthentication:JWT TokenRefresh TokenIdentityRole-based ClaimsCommunication:Fluent MailTwilio SMS