* Worked within an Agile/ Kanban environment and helped prioritize and scope feature requests. * Participated in requirements gathering, business analysis, and user meetings, discussing challenges to be resolved and converting user inputs into technical design and business intelligence reports. * Used SSRS to create parameterized reports, drill down reports, and ad hoc reports. * Used MDX to create calculations and datasets. Expertise in automating, delivering, and scheduling Power BI reports. Ability to test the data model, calculations, and analytical output quality. * Capable of creating highly visual reports, dashboards, and data narratives. * Created user groups with varying levels of access, created scheduled jobs.* Designed and built reports, dashboards, and templates that were easy to read and render.* Developed and implemented an NLP-based smart assistant using RASA NLU which dropped the wait times in half on Amazon's EC2 platform.* Kubeflow was used to deploy the data pipeline on Kubernetes.* Created data analysis reports using Matplotlib and Tableau.* Created a variety of data visualizations using Python and Tableau. * Created real-time data streaming solutions and batch-style large-scale distributed computing systems using Apache Spark, Spark Streaming, Kafka, and Flume.* Created data processing activities include receiving data from external sources, merging data, performing data enrichment, and loading data into target data destinations using PySpark. * Created a Schema RDD for JSON data to upload into Hive Tables to handle structured data. * Created a data services ecosystem architecture using Relational, NoSQL, and Big Data technologies. * Used pandas, numpy, sklearn, and plot.ly to quickly create models in Python. * These models are then implemented in SAS and interfaced with MSSQL databases, with updates scheduled on a regular basis for data visualization.

* Involved in building data pipelines that extract, classify, merge and deliver new insights on the data. * Worked on python and shell scripting to automate and schedule the workflows to run on Azure. * Experienced in working with UNIX/LINUX environments, writing UNIX shell scripts, Python. * Designed and build Modern data solutions using Azure PaaS service to support visualization of data. * Designed and built ETL pipelines to automate ingestion of structured and unstructured data.* Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks. * Advanced knowledge in performance troubleshooting and tuning of services in Azure like HD insight clusters, ADF, Databricks, Network. * Designed and implemented ADHOC data pulls using Azure Data Factory from on-premise onto SQL DW. * Developed Spark jobs using Pyspark and Spark-SQL for data extraction, transformation & aggregation from multiple file formats.* Implemented various Data Modeling techniques for CosmoDB. Create develop and test environments of different applications by provisioning Kubernetes clusters on AWS using Docker, Ansible, and Terraform.* Hands-on experience in setting up Apache Hadoop and Cloudera CDH clusters on Linux distributions environments. * Responsible for implementing monitoring solutions in Ansible, Terraform, Docker, and Jenkins. Automate Datadog Dashboards with the stack through Terraform Scripts. * Developed reporting applications for generating day to day reports for business. * Involved in performance tuning of Spark Applications for setting right level of Parallelism and memory tuning. * Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the R&D team.* Used optimization techniques in spark like Data Serialization and Broadcasting.

* Used Apache KAFKA for streaming services and data Manipulation as well as real-time synchronization. * Hands-on experience with Data Science Practices using Python language and tools like Python Scikit, Apache Spark, Mongo DB etc.* Experience with multiple data stores like AWS Redshift, AWS S3, AWS Athena, Druid, MongoDB. * Developed and deployed Scala on spark application for data analysis and aggregation. * Familiarity with Hadoop, Map Reduce . * Actively involved in deployment process using Jenkins, AWS EMR and AWS EC2.* Familiarity with source control tool GIT. * Developed and maintained data pipelines on ETL processes on Apache airflow and AWS data pipeline. * Worked fine-tuning mechanisms for data landing process as well as the queries to complete a set of given jobs or tasks in optimal time.* Developed interactive and intuitive year end dashboards and reports for internal reporting with node JS framework with druid and SQL database.

* Worked in Agile Methodology projects extensively and worked closely with SCRUM team in delivering agreed user stories on time for every Sprint by participating in SDLC process. * Developed Microservices by creating REST APIs and used them to access data from different suppliers and to gather network traffic data from servers.* Wrote and executed various NOSQL database queries from python using Python-NOSQL connector and NOSQL dB package. * Migrated the Django database to NOSQL with complete data integrity and designed, developed, and deployed CSV Parsing using the big data approach on AWS EC2. * Wrote MapReduce programs in python using Hadoop Streaming API and initiated the benefits and values for customer with big data as a service making the cost savings and agility for the customers and faster time to insights to maintain the Architecture.* Extracted Real time feed using Kafka and Spark Streaming and convert it to RDD and process data in the form of Data Frame and save the data as Parquet format in HDFS. * Treated code discrepancies using AWS Elastic Beanstalk to deploy code changes facilitating future platform growth, evolution and minimizing the risk of knock-on effect on the existing functionality. * Utilized Terraform to provision AWS resources (SNS, Lambda, SQS, CloudWatch Events, DynamoDB) to build Cloud hosted event-driven architecture which enables teams to process Claims events. * Provided leadership as a subject matter expert (SME) on hosting issues, clients and staff logins and general upgrades/ maintenance of servers

* Requirement discussions, design the solution. Estimated the Hadoop cluster requirements. * Responsible for choosing the Hadoop components (hive, pig, map-reduce, Sqoop, flume etc). * Responsible for building scalable distributed data solutions using Hadoop. Hadoop cluster building and ingestion of data using Sqoop. Imported streaming logs to HDFS through Flume. * Used Flume to collect, aggregate, and store the web log data from different sources like web servers, mobile and network devices and pushed to HDFS. * Developed Use cases and Technical prototyping for implementing Hive,and Pig. * Worked in analyzing data using Hive, Pig and custom MapReduce programs in Java. * Implemented partitioning, dynamic partitions and buckets in HIVE. * Installed and configured Hive, Sqoop, Flume, Oozie on the Hadoop cluster. * Involved in scheduling Oozie workflow engine to run multiple Hive and Pig jobs.