• Designing & implementing back-end application component in Apache NiFi for real-time data flow generating from IoT sensors.• Experience in database technology like SQL and document databases like MongoDb.• Agile software development practice.• Processing real-time data by working on NiFi data Pipeline; building Nifi data pipelines and ensuring data accuracy• Converting data into relevant insights (of application side) by means of data analytics and clear interpretations using data governance and data quality processes and Spark Scala• Writing Spark-Streaming applications to consume data from Kafka topics; writing the processed streams to Azure Redis Cache• Leveraging MongoDB to design and develop Application leveraging • Streaming components to support the intraday and real-time data processing through Apache Spark 2.0 and Spark SQL• Setting up Continuous Integration Deployment CI/CD pipeline by utilizing Azure DevOps• Managing activities like project management, requirement gathering also planning & supervising the daily activities of the application support• Managing, building, and optimizing modern data architectures• Participating in large-scale client engagements• Understanding business & technical requirements and implementing big data engineering techniques • Working on multiple data formats such as Parquet, Avro, JSON, XML, so on• Interacting using scrum calls daily for project analysis specifications and development aspects• Leading the delivery of innovative and engaging big data solutions• Conducting data discovery activities, performing root cause analysis, and recommending on data quality issues remediation

• Implemented:o Spark SQL Queries (Data Frame) in the spark applications o Multi-threading concepts using future concurrent parallel executiono Functional programming approach in spark applications• Administered the spark job applications using Ambari Console• Monitored & tested big data with Jupiter Notebook• Directed database administration for configuration, performance tuning, and maintenance with strong troubleshooting skills• Supervised customer interactions & led the developers for requirements understanding, technical analysis, estimation, designing of modules based on the functional aspects, by developing & converting into working quality of entire functionality to accomplish business cause/demands• Reviewed ETL process testing, scheduler design, and data extracts/transforms

• Used Pentaho Data Integration/Kettle to design all ETL processes to extract data from various sources including live system and external files, cleanse and then load the data into target data warehouse.• Created transformations that involve configuring the following steps: Table input, Table output, Text file output, CSV file input, Insert/Update, add constants, Filter, Value Mapper, Stream lookup, join rows, Merge join, Sort rows, Database Lookup, Set Environment Variables.• Created and saved Pentaho jobs in enterprise repository and scheduled them to run in production on weekly basis• Used dimension lookup/update step to populate data into SCDs.• Skilled in High Level Design of ETL DTS Package for integrating data from heterogeneous sources (Excel, CSV, Oracle, MySQL, flat file, Text Format Data).• Created Data Flow Mappings to extract data from source system and Loading to Target.