Utkarsh R. Email & Phone Number

• Develop MapReduce programs in Java for parsing the raw data and populating staging tables. • Write Flume configuration files for importing streaming log data into HBase with Flume. • Import several transactional logs from web servers with Flume to ingest the data into HDFS. • Use Flume and Spooling directory for loading the data from local file system (LFS) to HDFS. • Install and configure Pig, write Pig Latin scripts to convert the data from text file to Avro format. • Create partitioned Hive tables and work on them using Hive QL.• Extract, transform and load data from source systems to Azure Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL, and U-SQL of Azure Data Lake Analytics. • Data ingestion to one or more Azure services (Azure Data Lake, Azure Storage, Azure SQL DB, Azure SQL DW), and processing the data in Azure Databricks. • Design and architect scalable data processing and analytics solutions, including technical feasibility, integration and development for Big Data storage, processing and consumption of Azure data, analytics, big data (Hadoop, Spark), business intelligence (Reporting Services, Power BI), NoSQL, HDInsight, Stream Analytics, Data Factory, Event Hubs and Notification Hubs.

• Designed and build infrastructure for the Google Cloud environment from scratch. • Launched multi-node Kubernetes cluster in Google Kubernetes Engine (GKE) and migrate the dockized application from AWS to GCP. • Performed PoC to check various cloud offerings including Google Cloud Platform (GCP). • Utilized Stackdriver for monitoring, logging, compute engine and Dataproc. • Build data pipelines using Airflow in GCP for ETL related jobs using different Airflow operators. • Experience in installing and configuring Airflow in order to create and manage workflows using Python.• Worked on Workflow management including different task setups taking care of dependencies as well as automation using Apache Airflow. • Gained experience in Google Cloud Dataproc, Cloud Storage, Cloud Functions and BigQuery. • Used GCP Console to monitor Dataproc cluster and jobs, Stackdriver to monitor dashboards and do performance tuning and optimization of jobs which are memory intensive and provide L3 support for the applications in production environment.

• Work on Hadoop clusters which range from 4-8 nodes during pre-production stage and sometimes extended up to 24 nodes during production. • Implement bucketing and partitioning using Hive to assist the users with data analysis. • Design Oozie scripts for deployment of applications, with Perforce as the secure versioning software. • Implement partitioning, dynamic partitions and buckets in Hive. • Design changes to transform current Hadoop jobs to HBase. • Extend the functionality of Hive with custom UDFs and UDAFs.• Optimize performance using distributed cache for small datasets, partitioning, bucketing in Hive and Map-side joins. • Gained expertise in creating Hive UDFs using Java to analyze the data efficiently. • Load and transform large sets of structured, semi-structured and unstructured data and analyze them by running Hive queries. • Process image data through the Hadoop distributed system by using MapReduce and then store into HDFS. • Gained strong understanding of AWS components such as EC2 and S3.

• Engage in Functional testing, Integration testing, Regression testing, Smoke testing and Performance testing of Hadoop MapReduce workflows developed using Python, Pig and Hive.• Design predictive models to evaluate the Customer/Seller health score using Python scripts, perform computations and integrate with the Tableau Viz. • Gained experience in working with the Hadoop ecosystem through components like Yarn, HDFS and HBase.• Deploy and manage policies for S3 buckets and use them for storage and backup on AWS. • Migration of legacy data from SQL Server and Teradata into Amazon S3, handling data mismatch and loading of historical data.• Design custom SQL queries to verify the dependencies between daily, weekly, and monthly jobs.• Utilize complex views and monitor using database metrics to reduce the running time of complex queries. • Export data using Staging Tables to load data from different files. • Design SQL scripts to upload, retrieve, manipulate, and handle sensitive data in Teradata and SQL Server Management Studio.