Software Engineer
Current• Designed and Developed a Spring Boot program in Java that retrieves real time data from Kafka using Spark and executes Collibra Data Quality Checks in real time and separates the bad records.• Planned, Designed, Developed, Tested and Deployed multiple Spring Boot programs in Java that aimed at ingesting metadata from SAS, Denodo, Oracle, etc. into Collibra in form of assets, attributes and relationships to form complete metadata lineage.• Developed and Optimized Collibra workflows using flowable in Groovy that helped in authorizing access to Collibra Datasets.• Built multiple Rulesets for various checks in MuleSoft Anypoint that facilitates best practices during API calls.• Built pipeline in Azure Data factory that stored Collibra Insights data into Azure SQL. Built Tableau dashboards that used Insights Data stored in Azure SQL to provide useful information and statistics in Collibra DGC Dashboards.• Developed Gitlab CI/CD pipeline that pulled required images securely from a public repository and pushed the images onto the required private registry. The pipeline was developed with reusability in mind that allows it to work for any number of images.
