Senior Data Engineer
Washington D.C. Metro Area
• Experienced in working with Hadoop clusters using Cloudera distributions.• Developing big data solutions in Hadoop Ecosystem using Apache Spark, Hive and Pig.• Developed various data ingestion pipelines using streaming tools like Flume, Spark and Kafka.• Worked with Data lake architecture and setting up Hadoop environment foringesting claim and provider data from multiple sources.• Import/export data using Sqoop from Relational Database systems to HDFS.• Involved in creating Hive tables, loading and analyzing data using hive queries.• Worked on Spark SQL, Data Frame, Spark Streaming, Reading/ Writing data from Avro, Json, Text,Parquet to RDD.• Exploring with Spark various modules of Spark and working with DataFrames, RDD and Spark Context.• Serve as a mentor and trainer of other members of the team. Provide guidance, assist in building skill sets, and provide knowledge of ETLprocessing, T-SQL debugging, Performance tuning technique, SSIS, SSRS for junior IT operation specialist.