Shrivani N Email and Phone Number

• Involved in Requirement gathering, Business Analysis and translated business requirements into technical design in Hadoop and Big Data.• Worked on SQOOP implementation which helps in loading data from various RDBMS sources to Hadoop systems.• Developed Python scripts to extract the data from the web server output files to load into HDFS.• Worked on HBASE setup and storing data into HBASE, which will be used for further analysis.• Worked on Written a python script which automates to launch the EMR cluster and configures the Hadoop applications using boto3.• Created various data pipelines using Spark and Spark SQL for faster processing of data.• Designed number of partitions and replication factor for Kafka topics based on business requirements and worked on migrating MapReduce programs into Spark transformations using Spark. initially done using python (PySpark).• Implemented Spark using Python and Spark SQL for faster processing of data and worked on migrating MapReduce programs into Spark transformations using Spark and Scala, initially done using python (PySpark).

• Worked on Tensorflow, Keras, NumPy, Scikit-Learn, tf.Data API, Jypyter Notebook, in Python at various stages for developing, maintaining and optimizing machine learning model.• Extracted Fingerprint image Data stored on local network to Conduct Exploratory Data analysis (EDA), Cleaning and organize. Ran NFIQ algorithm to ensure data quality by collecting the high score images. Finally Created histograms to compare distributions of different datasets.• Transformed the image dataset to protocol buffers, serialized and finally stored inside TFrecord data format.• Loaded the data in GPU and achieved Half Precision FP16 training on Nvidia Titan RTX and Titan V GPU for TensorFlow 1.14.• Specialized in transforming data into user-friendly visualization to give business users a complete view of their business using Power BI. • Used various sources to pull data into Power BI such as SQL Server, oracle. • Involved in installation of Power BI Report Server. • Developed reports to visually explore data and create an interactive report Excel and Power BI. • Using a query editor in Power BI performed certain operations like fetching data from different file.• Optimized TFRecord data ingestion pipeline using tf.Data API and made them scalable by streaming over network, thus enabling training of models with Datasets which were bigger than CPU memory.• Automated training and optimization of model hyperparameters to quickly conduct and test 50 different variations of the model. Finally storing the results and generating automated reports.• Maintaining Models created by other data scientists, retrained them with different variations of datasets.• Created tooling for other data scientists to help them become more effective at exploring the data and other tasks.

• Developed a pipeline using Hive (HQL) to retrieve the data from Hadoop cluster, SQL to retrieve data from Oracle database and used ETL for data transformation.• Analyzed and gathered business requirements from clients, conceptualized solutions with technical architects, and verified approaches with appropriate stakeholders, developed E2E scenarios for building the application.• Derived data from relational databases to perform complex data manipulations and conducted extensive data checks to ensure data quality. Performed Data wrangling to clean, transform and reshape the data utilizing NumPy and Pandas library.• We have worked with datasets of varying degrees of size and complexity including both structured and unstructured data and Participated in all phases of Data mining, Data cleaning, Data collection, variable selection, feature engineering, developing models, Validation, Visualization and Performed Gap analysis.• Performed end-to-end Architecture, and implementation assessment of various AWS services like Amazon EMR, Redshift, S3, Athena, Glue, and Kinesis.• Optimized a lot of SQL statements and PL/SQL blocks by analyzing the execute plans of SQL statements and created and modified triggers, SQL queries, stored procedures for performance improvement.• Implemented Predictive analytics and machine learning algorithms in Data bricks to forecast key metrics in the form of designed dashboards on AWS (S3/EC2) and Django platform for the company’s core business.• Participated in features engineering such as feature generating, PCA, Feature normalization and label encoding with Scikit-learn preprocessing. Data Imputation using variant methods in the Scikit-learn package in Python.• Used Sqoop to move data from oracle database into hive by creating a delimiter separated files and using these files in an external location to be used as an external table in hive and further moving the data into refined tables as parquet format using hive queries.