•Built a backend server application from scratch using Flask (micro web framework).•Utilised Airflow to schedule and organise workflows that pull million records of raw data from the data lake on a frequent schedule and alter it into an informative and submittable standard format.•Implemented parallel data processing techniques such as threading and multi-pod runs using k8s.•Have leveraged Kafka in order to share data with downstream applications by producing data to a Kafka stream that the downstream applications can consume.•Developed a global utility package that can be pip-installed and used as a package in any other repository, reducing utility module duplication in each repository.•Utilised GitHub Actions to integrate the test case validations, packaging and deployment to k8s.•Tools Used: Python, Flask, PostgreSQL, AWS Cloud Services, Airflow, Kafka, GitHub.

•Developed Data Quality checks(DQ) using Python & Pandas library which helps to identify discrepant lab data.•Developed shell scripting and Python programs to automate the errors extraction from Log files and segregating it for quick analysis and RCA. •Implemented Open AI to generate the DQ code by using appropriate prompts and pseudo code of the DQ description.•Modified the existing vulnerable code to an efficient, flexible and modular code. Modified code was ensured to follow the PEP standards.•Tools Used: Python, PostgreSQL, GitHub.