Software Architect
CurrentAccountable to ingest, transform and load data using Spark/Hadoop framework by ingesting over 10 million data every day from over 100 different sourcesBuilt data warehousing application to Enhance, Transform and Load data and send it to cubes for target applications.Responsible for managing/maintaining Production Cloudera/ETL systems and DWH systems and managing daily failures within SLAManaging/Leading team with Spark, Hive and Kafka based loads and Oracle Data Integrator/Infomatica for DWH applicationsDesigning for entire Batch/Streaming and DWH loads for Vodafone ItalyEnsuring successful production deployments and closely monitoring multiple projects under Vodafone ItalyMonitoring sizing, mapping and creation of target data lake architecture and designExtending Project Management support to manage implementation of the big data lake architecture, technology and; tools incollaboration with the vendorParticipating in the design, development and testing of the data components needed to support clients and reporting requirements and to integrate these designs into the overall client data strategyIdentifying and analyzing business requirements, as well as integration issues and offering solutionsExtensively working on ETL tool/complex ETL processes and analyzing complex external data for mapping the productDesigned, developed and deployed an entire solution when there was a performance bottleneck in production system with 200 million records streaming everydayDeployed solution within the SLA and increased efficiency by listening incoming streaming messages close to 250 million and ingesting, transforming and loading them to target tables within 40 minutes; this was very highly appreciated by the customer as it was a crucial revenue report score which need to be published everydayPerformed in highly challenging and liberty production war rooms by designing and architecting solutions during serious issues in Production systems
