As a part of Data Enrichment Team need to analyze, design and build E2E data flow pipelines for a streaming product to process two million records per second. We get the ingested data in Azure event hub and process the data using Spark Streaming and Scala and put the final data to the Kafka topic, Grafana is used for visualization. For CICD purpose Octopus and git has been used.

As a part of Datamart project, we need to analyze and build the fact and dimensions table using the source data coming through sql server. The fact tables were containing aggregated metrics that were used for KPI calculation in power BI dashboard. Star schema was used as the design and development process. There were multiple ETL pipelines developed like schema modeler, dimension and fact modeler to build the lineage of data flow.

Implemented reusable, modular components using React.js to improve code maintainability and scalability. Utilized modern JavaScript frameworks and libraries (e.g., Redux, Next.js) to enhance application functionality and maintainability. Implemented lazy loading for images and components, reducing load times and enhancing the user experience. Collaborated with design, product & back-end teammates to build, evaluate and maintain platform features. Wrote unit and integration tests using Jest and React Testing Library to ensure code reliability and minimize bugs.