Role of a pod lead for a dedicated team in managing capacity and service sustainability of an estate with 70 odd clusters with over 2500+ servers, identifying and strategizing capacity optimizations through various initiatives rendering huge cost savings to the bank by coordinating with senior stakeholdersKey responsibilities• Virtualization of control nodes for cluster efficiency and re-use large commodity nodes as Data nodes for storage and processing enhancement• Storage optimization through server disk upgrades from 4 to 14T (Huge cost saving)• Data centre planning and management for Bigdata estate• ESX provisioning and engaging with VMware team to build customized virtual machines• Ansible automated Hadoop Node build and repurpose for HP, Lenovo and Dell nodes with various Operating System versions• New server procurements and build for evergreening from HDP to CDP and replacing RHEL 6 with RHEL 7 Infra• Data analytics and visualization• Stakeholder engagement from various GB/GFs and senior management to discuss project road map/progress and propose insightful team strategies• Infra management and server provisioning from various vendors like HP, Lenovo, DELL, etc

Working on analysing the data on clusters and building powerful insights to maintain data integrity, deduplication, consistency, etc and optimizing cluster storage and memory resources through a fantastic analytical engine built from scratch using ETL through hive/spark, ansible and visualization on Grafana and Qlik sense.Key Responsibilities• Data analytics to achieve Data Integrity and consistency• Data deduplication• Data compression• Build and maintain an inhouse tool called Bigdata 360 that analyses data across clusters and draws meaningful insights thus generating huge benefits through optimizing data storage and processing. This also was an added benefit to the key business stakeholders who otherwise would have to spend high costs without this data cataloguing tool• Build and maintain a super-efficient, robust, transparent and self-explanatory chargeback tool which scans through clusters to capture data, tools, licenses and resources and charges the stakeholders aptly• Install Elastic Search and design Kibana and Grafana dashboards on Hadoop cluster for performance monitoring and optimization by providing fine tuning insights

Responsible for administration of Hadoop infrastructure consisting of over 2500+ servers spread across over 70 clusters. Build and deploy new data platform and data flow clusters using Ansible/Terraform. Build and maintain clusters with IBM CDC, Kafka, NIFI and any other streaming requirements. Engineering cloud-based solutions for data migrations from on-prem to GCS. Key Responsibilities• Installation and configuration of new HDP with additional third-party tooling like Python, Datameer, Platfora, SAS,etc and HDF Clusters with Kafka and NIFI installations• Hadoop patch updates, version upgrades when required. • Production Support, Projects Support, Service Management. ➢ Process Improvement, setting up support practices and delivery POD’s/SCRUM’s ➢ Create Knowledge Base and Best Practices Design Documents. • Conduct Proof of concepts and Product Evaluations ex: Unravel, STRIIM, Superset, etc• Technical Escalation point of contact for Global Banking & Markets • Liaise with other Teams & Design SDD for projects ex: Storage, Network, Monitoring etc. ➢ SSL setup for Schema registry/Ranger Kafka Plugin/Ambari ➢ Install Elastic Search and Kibana on Hadoop cluster

Worked in a complex 24*7 production support team to a pre-mature version of Hadoop (1.7) with numerous issues and bugs providing great value through various automations for self-healing of incident tickets thus bringing down the tickets from 3000+ to 100 within a span of 4 months. The role included co-ordination with vendor support teams(Hortonworks) who themselves benefitted hugely through our root causes and recommendations and evolved the product overall, since we were one of the biggest clients.

Worked in providing monitoring solutions for servers and networks, which monitor infra, processes, filesystems, DBs, logs, WebSphere applications, etc. The role also including in redirecting alerts from BMC patrol , control M, etc to an incident generation tool which is then enriched in a tool called BEM to hit respective support GSD queues for immediate attention and action.