Data Engineer
Current➢ Analyze and understand the business stories/requirements to convert into technical design document.➢ Identifying, designing, and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes that is highly scalable and re-usable using Snowpipes and DBT Jobs.➢ Designed and Developed DBT jobs and Migrating DataStage processes to DBT to ingest data into Snowflake platform.➢ Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using Snowpipe, FiveTran, DBT and SQL technologies into Snowflake system (loading into Raw, Structured, Curated, Analytics and Reporting Schema).➢ Implemented multiple different automation and re-usable scripts to ease the work of other engineers, save the development cost/efforts and simplify maintenance and testing.➢ Built tableau dashboards to slice and dice the data to take business critical decisions.➢ Performing POC on numerous methodologies and selecting the appropriate technology that suits company’s need and standards.➢ Hands on experience on Azure data lake and designed and developed numerous methodologies to load source files into ADLS.➢ Enhance/Upgrade DataStage jobs as per business review comments and current requirements.➢ Data Quality assurance to deliver a 0-defect product by building Datagap jobs.➢ Working with stakeholders including data, design, product and Business teams and assisting them with data-related technical issues.➢ Troubleshooting technical issue that are raised as Incidents/Tickets.➢ Performance Engineering of DB SQL programs and ETLs.➢ Coded complex SQLs (ELT scripts) to load data into Data Warehouse Foundation/Aggregate tables.
