• Building Airflow data pipelines for ingestion and transformation with Python, EMR, Redshift, and S3 using APIs for Salesforce and Gainsight (Customer Success) data sources. Creating KPI semantic views for Tableau to use • Create dimensional modeling for upstream tables.• Research, propose, and implement conversion to Databricks from Redshift based on Delta Tables.

Workforce Intelligence Recommendation: (Amazon) 11/2020 – 7/2021• Building pipelines for Amazon internal candidate and site location data, and external labor force data using Redshift, Glue, SageMaker, and Airflow. • Generating nearby site pursuits based on commute time regions using MapBox and ArcGIS isocrhone REST APIs for SageMaker to build ML models.• Publishing QuickSight with labor potential, competitor insight, and candidate intelligence based on SageMaker ML models and external Census data.

• Designed and built data pipelines for a headcount analytics platform. Combining headcount, recruitment and hierarchical data to post to Essbase from Teradata.• Airflow integration for the hardware activation data when triggering from email events to support Tableau and Tableau Prep down streams.

Built Presto data pipelines from MySQL, Oracle, and in-memory data objects. Created KPI metrics backends and dashboards. Reconciled data between Oracle and Presto.

• Enhancing Hive queries to improve the eligibility pool of loan candidates by analyzing historical credit activities. Developing Lambda and DynamoDB codes for AWS migration.• Building a machine learning pipeline to analyze model performance by using Spark Python to extract data from a REST API and load flatten confidence scores into Hive; Developing Scala codes to detect and assign inter account transfer for revenue calculation.• Building metrics Hive backends based on KPI data models for Marketing and Finance Tableau dashboards – monitoring loan performance and revenue. Populate Hive tables with UDF generated by H2O propensity score matching models.

Big Data Analytics• Successfully supported and secured a TPCx-BB top rank for the Hadoop benchmark publication for Dell 14G servers and networking in the Scale Factor 10000 category. Tasks included tuning Hive configurations, Spark Machine Learning Library with MKL (Intel Math Kernel Library), validated with Hive explain plans, and performed Hive on Tez characterization. Developed automated performance codes (TPCx-BB, IO characterization, and network throughput) with Python.• Validated and designed on-prem BDaaS (Big Data as a Service) for BlueData. Performance characterized BlueData docker multi-tenancy for Hadoop, Cassandra, and R-Studio. Compared NVIDIA Tesla V100 and Pascal P100 GPU vs CPU for MapD databases and TensorFlow. Developed Python codes to drive tests and collect results.IOT and Analytical Insight Modules• IOT data ingestion and ETL: Parsed and flattened MongoDB JSON files to load into Impala. Involved in development of predictive maintenance model for IOT devices.• Executed scale-out performance for the EMC Analytical Insight Module in the Isilon Data Lake environment.Hadoop and NoSQL• Automated scale-out performance and tuning Hadoop performance for Isilon OneFS/WebHDFS.• Produced competitive studies for Greenplum, Cassandra under different configurations including virtual and bare metal.• Automated and developed python scripts to drive workloads for TPC benchmark, Terasort and Cassandra tests; Generating ETL code scripts and datasets.Virtualization of Greenplum Appliance• Completed Oracle code conversion and data migration to virtualized Greenplum for EMC DWBI platform. Utilized open source and customized Perl scripts to convert Oracle stored packages and DDLs, extract data from Oracle, and load into Greenplum.• Presented at EMC World.Flash Drive/SSD Characterization for Data Warehouse, Hadoop and OLTP• Performance analyzed and characterized for DSS, Analytics, ETL and OLTP workloads.

Migrated encrypted sensitive token data from RSA to VeriSign platform.

 Data Services, ETL, and Web Mining: Provide internet panelists activity data to the clients and inter-company divisions.- Process data under Netezza and Oracle.- Desing database structure and file layouts.- Extract and create files using Java, sh and SQL.- Transport and notify through TIBCO messaging.  Analytical Platform for Internet Panlist Behavior: Design and develop OLAP solutions under Netezza to supply panelists’ surfing activity data. Leveraging Cognos for additional analytical functions and export features.- Batch process matching url patterns to track internet activity of the panelists.- Data copying from Oracle and other data sources.Validate, develop and design reports for custom requirement.