Lead Data Engineer - Big Data Analytics
Current•Built ETL pipelines to automate ingestion of transactional data for FODL UTV Spend & Non-Spend usecase to provide a standardized view of finance analytics for discount Business volume, discount billed business & discount revenue.•Developed, refined, and scaled data processing pipelines for management reporting.•Translated the existing IBI Queries to Spark SQL/Dataframe with Performance Optimization and reduced query execution time by 30%.•Periodic Evaluation of new technology through developing POC.•Providing technical guidance and mentorship to team members to enhance their skills and performance.•Build an enterprise dashboard for corporate portfolio (HR, Finance, Dependent departments) to enable data for business decisions & data science models.•Designed and built ETL pipelines to automate ingestion of structured and unstructured data from various external systems into foundry to deliver complete solution for various analytics. This will enable visual exploration and custom dashboarding.•Automated ETL processes making it easier to wrangle data and reduced time by 20%.•Managed, mentored and grew a team of big data engineers.