Data Engineer
CurrentData Engineer: Apace Hadoop Framework (HDFS, MapReduce, YARN). Perform data analytics using Pig Hive. Data loading using Sqoop and Flume. Apache Spark Framework (Spark SQL, Spark Streaming). Data Warehouse using Talend Data Integration tool. Experience working with AWS technology stack, including Amazon RDS and Aurora for relational database, Amazon DynamoDB for NOSQL database. Amazon Redshift for data warehouse. Amazon Kinesis Firehose to transform and load streaming data into Amazon Redshift.Project 1: Big Data and Hadoop Project 2: Apache Spark and ScalaProject 3: Configure and deploy a Redshift cluster, loading data into it, and query data using an external SQL client. Used Amazon Kinesis Firehose to transform and load streaming data into Amazon Redshift.