Dipanjan S. Email and Phone Number

Some of my recent work includes while building a framework -■ Batch and streaming data ingestion■ Automatic generation of Slowly Changing Dimensions (SCD1 andSCD2) - Out of the box■ Fully Configuration options for various data loading strategies (full load,delta load, append, merge, delete)■ Support for a no-code approach using Python / SQL■ Data pipeline generation for raw, processed, and curated data layers - This happen out of the box■ Integration with various data sources (Event Hub, Kafka, ADLS) anddestinations (Event Hub, Service Bus, Cosmos db, neo4j etc)■ Logging to Elasticsearch and Microsoft Teams channels - Real time observability■ Build a capability to bring your own app wherein the no code approach isn't fitting for a complex use case.■ Build an API layer so that end user can communicate/leverage above features.