Rupak Biswas Email and Phone Number

Working as Data engineer for a US-based insurance client responsible for developing and managing data flows using Azure Databricks and Azure Data Factory, Implementation of new Databricks features such as Autoloader, Unity Catalog, and SQL Warehouse.

Working as a data engineer for a UK based manufacturing client related to railways. I am responsible for working on different use cases which are responsible for data flow of different weather records, which will be later used to determine track conditions for train run and help in issue detection.Need to create a flow for each parameters to ingest into our database then populate it to SQL DB, which where the UI team will pull the data. The source files are in grip2 format so need to convert then to parquet and then load the data into delta tables and pull the latest data from them after some manipulations. This data will be received every hour which we need to load into SQL DB within 15 min so as to get the data ready for users to look into. The data we are receiving is of hourly basis for +-10days.

Working as a data engineer in R&D. I was responsible for working on different use cases along withdifferent data requests from DataScience team and Development teams. The primary use case which I worked on is Query Builder,where primarily I needed to do aPOC on the design and end to end flow. This included ingesting data from Adls from developmentenvironment to R&D environmentand create delta tables for the same. I used jdbc connection to read the data. Tools used wereDatabricks, SSMS and sql warehouse

Worked within an agile team as a Data Engineer for a major UK based retail client.1. Responsible for the work flow of the various CI/CD prebuild pipelines. Working on Azure Bicep andAzure Portal formaintaining environments, access control, etc.2. Working on Azure services such as Resource groups, Azure Active Directory, Subscriptions, KeyVaults, Virtual Machines, etc

Worked within an agile team as a data engineer for a major UK based retail client.1. Experience of developing and enhancing etl work flows manly consisting of ingestion, provision usingcsv, xml files, apis, kafka topicswithmultiple data clients, such as salesforce, wcs, medallia, beanstore, bazzarvoice, etc.2. Worked with files based on encryption and decryption from ingest to present layer. Have gatheredworking knowledge onSynapse as well.3. Developed feature for ingesting data from query APIs, complex xml files and streaming data usingconfluent Kafka topics

I had experience of overall etl work flows.1. Responsible for the etl work flows being delivered properly and effectively.2. Using monitoring tools as Datastage, Controlm, Dbeaver, Kibana.3. Working on technologies like Plsql, shell scripting to identify issues, to establish solutions quickly andefficiently with clarity.4. Build critical flows to track the process. And had revived and updated multiple scripts as perrequirement.