Karishma M Email & Phone Number

New York, Ny, Us

• Work on requirements gathering, analysis and designing of the systems.• Developed Spark programs using Scala to compare the performance of Spark with Hive and SparkSQL.• Developed spark streaming application to consume JSON messages from Kafka and perform transformations.• Involved in monitoring Big query, Dataproc and cloud Data flow jobs via Stack driver for all the different environments.• Consulting with client business community to gather requirements for specific BI scorecard, vdashboard and reporting needs.• Wrote detailed specifications to be used in developing reports and processes. • Worked with product teams to create various store level metrics and supporting data pipelines written in GCP’s bigdata stack.• Used Sqoop import/export to ingest raw data into Cloud Storage by spinning up Cloud Dataproc cluster.• Developed dashboard prototypes using Cloud Dashboard Tools Looker and AWS Quicksight managing all aspects of the technical development.• Preparing design documents for changes to be implemented, performing peer reviews and developing scripts to load and process data. • Applying technical skills using Python, VBA, Tableau, Power BI, Essbase and SQL in data collection, data analysis and reporting to procure data from database structures and provide Business Intelligence solutions to client requests in a timely manner. • Worked on migrating an entire oracle database to BigQuery and using of power bi for reporting.• Worked on moving data between GCP and Azure using Azure Data Factory.• Created and maintain the data pipelines using Matillion ETL and Fivetran.• Created measure and dimensions on Looker based on the data and the business requirements.• Converted PL/SQL type of code to both bigquery - python architecture as well as azure databricks and pyspark in dataproc.

Woonsocket, Ri, Us

• Work on requirements gathering, analysis and designing of the systems.• Actively involved in designing Hadoop ecosystem pipeline. • Developed Spark code using Scala and Spark-SQL/Streaming for faster testing and processing of data. • Involved in designing Kafka for multi data center cluster and monitoring it. • Responsible for importing real time data to pull the data from sources to Kafka clusters. • Worked with spark techniques like refreshing the table and handling parallelly and modifying the spark defaults for performance tuning. • Involved in migrating on prem Hadoop system to using GCP (Google Cloud Platform).• Deploying Spark cluster and running PySpark jobs using DataProc and scheduling batch jobs through Cloud.• Responsible for implementation and ongoing administration of Hadoop infrastructure• Implemented Spark RDD transformations to Map business analysis and apply actions on top of transformations. • Involved in migrating MapReduce jobs into Spark jobs and used SparkSQL and Data frames API to load structured data into Spark clusters. • Involved in using Spark API over Hadoop YARN as execution engine for data analytics using Hive and submitted the data to BI team for generating reports, after the processing and analyzing of data in Spark SQL. • Performed SQL Joins among Hive tables to get input for Spark batch process. • Worked with data science team to build statistical model with Spark MLLIB and PySpark.

• Analyzing Functional Specifications Based on Project Requirement.• Ingested data from various data sources into Hadoop HDFS/Hive Tables using SQOOP, Flume, Kafka.• Extended Hive core functionality by writing custom UDFs using Java.• Developing Hive Queries for the user requirement. • Worked on multiple POCs in Implementing Data Lake for Multiple Data Sources ranging from TeamCenter, SAP, Workday, Machine logs.• Developed Spark code using Scala and Spark-SQL/Streaming for faster testing and processing of data.• Worked on MS Sql Server PDW migration for MSBI warehouse.• Planning, scheduling, and implementing Oracle to MS SQL server migrations for AMAT in house applications and tools.• Worked on Solr Search Engine to index incident reports data and developed dash boards in Banana Reporting tool.• Integrated Tableau with Hadoop data source for building dashboard to provide various insights on sales of the organization.• Worked on Spark in building BI reports using Tableau. Tableau was integrated with Spark using Spark-SQL. • Developed Spark jobs using Scala and Python on top of Yarn/MRv2 for interactive and Batch Analysis. • Created multi-node Hadoop and Spark clusters in AWS instances to generate terabytes of data and stored it in AWS HDFS.