• Data Engineering professional with solid foundational skills and proven tracks of implementation in a variety of data platforms. Self - motivated with a strong adherence to personal accountability in both individual and team scenarios.• Around 8 Years of experience in Data Engineering, Data Pipeline Design, Development and Implementation as a Sr. Data Engineer/Data Developer and Data Modeler.• Strong experience in Software Development Life Cycle (SDLC) including Requirements Analysis, Design Specification and Testing as per Cycle in both Waterfall and Agile methodologies• Expertise in using various Hadoop infrastructures such as Map Reduce, Pig, Hive, Zookeeper, Hbase, Sqoop, Oozie, Flume, Drill and spark for data storage and analysis.• Strong experience in writing scripts using Python API, PySpark API and Spark API for analyzing the data.• Proficient in Statistical Methodologies including Hypothetical Testing, ANOVA, Time Series, TEMP Principal Component Analysis, Factor Analysis, Cluster Analysis, Discriminant Analysis.• Skilled in System Analysis, E-R/Dimensional Data Modeling, Database Design and implementing RDBMS specific features.• Knowledge of working with Proof of Concepts (PoC's) and gap analysis and gathered necessary data for analysis from different sources, prepared data for data exploration using data munging and Teradata.• Well experience in Normalization and De-Normalization techniques for optimum performance in relational and dimensional database environments.• Expertise in Python and Scala, user-defined functions (UDF) for Hive and Pig using Python.• Experience in developing Map Reduce Programs using Apache Hadoop for analyzing the big data as per the requirement.• Experience in working with Flume and NiFi for loading log files into Hadoop.• Experience in working with NoSQL databases like HBase and Cassandra.• Experienced in creating shell scripts to push data loads from various sources from the edge nodes onto the HDFS.• Good Experience in implementing and orchestrating data pipelines using Oozie and Airflow.• Worked with Cloudera and Hortonworks distributions.• Good working knowledge of Amazon Web Services (AWS) Cloud Platform which includes services likeEC2, S3, VPC, ELB, IAM, DynamoDB, Cloud Front, Cloud Watch, Route 53, Elastic Beanstalk (EBS), Auto Scaling, Security Groups, EC2 Container Service (ECS), Code Commit, Code Pipeline, Code Build, Code Deploy, DynamoDB, Auto Scaling, Security Groups, Red shift, CloudWatch, CloudFormation, CloudTrail, Ops Works, Kinesis, IAM, SQS, SNS, SES.
