Big Data, Data Analytics, Warehouse, Etl, Aws Consultant
Current-> Selection tools and technology as per organizational data size -> System deployment (of selected big data tools) in on-premise (HDP/Cloudera) or in cloud (AWS) -> Scripts for data collection, preprocessing, normalization, enrichment, storage in HDFS/Kafka-> Spark job for analysing both batch and streaming data-> Display analysed data using visualization tool or customized portal (D3+react+node)-> Collecting data from Netflow,SFlow,Syslog using… Show more -> Selection tools and technology as per organizational data size -> System deployment (of selected big data tools) in on-premise (HDP/Cloudera) or in cloud (AWS) -> Scripts for data collection, preprocessing, normalization, enrichment, storage in HDFS/Kafka-> Spark job for analysing both batch and streaming data-> Display analysed data using visualization tool or customized portal (D3+react+node)-> Collecting data from Netflow,SFlow,Syslog using VFlow,custom scripts and Nifi-> Already implemented: Elastic Search,Lucene,HDFS,HIVE,Kafka,Flume,Sqoop,Spark,Scala,Nifi-> AWS & server Administrator Show less