Sridhar K Email & Phone Number

Involved in requirements gathering on the data subjects and analysis of the source data before designing.Build Hadoop Datalakes/Deltalake and developed the architecture and used in implementations within the organization and Ingest Legacy datasets into HDFS using Sqoop Scripts and populate Enterprise DataLake by importing tables from Oracle and other databases and Mainframe Sources and store them in partitioned hive tables using different file compression techniques. Installed and Setup Hadoop CDH clusters for development and production environment and installed and configured Hive, Pig, Sqoop, Flume, Cloudera manager and Oozie on the Hadoop cluster.Created data pipeline of gathering, cleaning and optimizing data using Hive, Spark and Building data pipeline ETLs for data movement to S3, then to Redshift and Developed automated data pipelines from various external data sources (web pages, API etc) to internal data warehouse (SQL server, AWS), then export to reporting tools.Implement AWS Data Lake leveraging S3, terraform, EC2, Lambda, and IAM in performing data processing and storage while writing complex SQL queries, analytical and aggregate functions on views in Snowflake data warehouse to develop near real time visualization using Tableau.Used Spark Streaming APIs to perform transformations and actions on the fly for building common learner data model which gets the data from Kafka in near real time and persist it to Cassandra.Responsible for writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL) and Hive UDF's in Python.Involved in installing EMR clusters on AWS and used AWS Data Pipeline to schedule an Amazon EMR cluster to clean and process web server logs stored in Amazon S3 bucket and Created monitors, alarms, notifications and logs for Lambda functions, Glue Jobs, EC2 hosts using Cloudwatch

Kansas City, Missouri, United States

Designed and developed architecture for data services ecosystem spanning Relational, NoSQL, and Bigdata technologies. Extracted Mega Data from Amazon Redshift, AWS, and Elastic Search engine using SQL Queries to create reports.Generation of business reports from DataLake using Hadoop SQL (Impala) as per the Business Needs and automation of Business reports using Bash scripts in UNIX on Datalake by sending them to business owners.Involved in start to end process of Hadoop jobs that used various technologies such as Sqoop, PIG, Hive, MapReduce, Spark and Shel lscripts (for scheduling of few jobs) extracted and loaded data into DataLake environment (AmazonS3) by using Sqoop which was accessed by business users and data scientists.Responsible to manage data coming from various sources and involved in HDFS maintenance and loading of structured and unstructured data and visualize the HDFS data to customer using BI tool with the help of Hive ODBC Driver.Created data pipeline for different events of ingestion, aggregation and load consumer response data in AWS S3 bucket into Hive external tables in HDFS location to serve as feed for tableau dashboards and responsible for creating on-demand tables on S3 files using Lambda Functions and AWS Glue using Python and PySpark.Analyzed the SQL scripts and designed it by using PySpark SQL for faster performance and encoded and decoded JSON objects using PySpark to create and modify the data frames in Apache Spark

Chicago, Illinois, United States

Involved in architecting system interfaces, understand interface requirement and design data model (Logical and Physical) using ERwin and Deliverables are PDM, DDL scripts and STTM documents. Provided Data Architecture solutions for multiple relational and dimensional models and involved in Data Warehouse Support –Star Schema and Dimensional modeling to help design DataMart and data warehouse. Imported Data from Different Relational Data Sources like RDBMS, Teradata to HDFS using Sqoop and imported Bulk Data into HBase Using Map Reduce programs and perform analytics on Time Series Data exists in HBase using HBaseAPI.Designed and implemented Incremental Imports into Hive tables and used Rest API to Access HBase data to perform analytics.Forward engineering the data model and creating the development Database modify and maintain all changes to the Data model and Database scripts.

India

Involved in architecting system interfaces, understand interface requirement and design data model (Logical and Physical) using ER Studio and Deliverables are PDM, DDL scripts and STTM documents. Performed Reverse Engineering of the legacy application using DDL scripts in ER Studio and developed Logical and Physical data models for Central Model consolidation.Defining data governance process: processes for data quality rules definition, review process, communication plan, templates, etc.Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables in the EDW.Involved in performing extensive Back-End Testing by writing SQL queries and PL/SQL stored procedures to extract the data from SQL Database using Hadoop