Worked for a semiconductor company as a contract worker. The company name is SK Hynix, it is the third place of the world of semiconductor companies.I designed developed a data-lake platform, it ensures that supporting accurate data from various sources to the final data store in order to provide people who want to analyze the data easily, efficiently and fastly.• Designed data pipelines and data-lake architecture.• Designed software architecture that collecting huge data from various equipment.• Developed ETL process using MapReduce and Apache Apark that accumulating manufacturing data to HBase• Designed and developed data indexing architecture using Apache Solr and ElasticSearch that data stored on HBase.Environment- Hadoop, Spark, Hive, HBase, Kafka, Apache Solr, Elasticsearch- MapReduce, Spark Dataframe - Linux, CentOS, Redhat- Java, Spring, Multi-Thread

Data Engineer at WiderplanetWiderplanet is an online advertisement company in South Korea. We deliver and recommend a marketing contents to an individual user that analyzed from bigdata which collected users' behavior online• Developed ETL process using Apache Kafka from various sources to data warehouses so that the next team• Developed ETL daemon using JAVA to store into Hadoop HDFS or Elastic Search.• Developed real-time aggregation process using Apache Kafka Streams.• Maintained huge clusters. Hadoop, Kafka, Elastic Search• Collected various data to provide to monitoring dashboards that the data from various systems, daemons such as Hadoop, Kafka, Elastic Search, Linux, ETC.Environment- Hadoop, Spark, Hive, HBase, Kafka, Redis- Linux, CentOS, Ubuntu- Java, Spring, AOP, Multi-Thread

Big Data DeveloperProject titleBig data Performance Management "Flamingo" (http://www.ex-em.com)Project DescriptionFlamingo is professional solutions for Hadoop performance management, manages efficient availability and performance for Hadoop eco-system.• Designed Flamingo project Database Schema, Software architecture.• In-depth knowledge of Hadoop Architecture and components such as NameNode, HDFS, Datanode, YARN, ResoruceManager, Map Reduce, NodeManager.• Developed an "HDFS Browser" that can handle over 10k files and directories in a web browser.• Used Salted Tables, Row Timestamp Columns in Apache Phoenix(https://phoenix.apache.org/) for increasing performance benefit in collecting data ingestion.• Integrated Apache Zeppelin(http://zeppelin.apache.org/) notebook• Used Zookeeper to provide coordination services to the cluster.Environment- Hadoop, Spark, Hive, HBase, Kafka, Workflow, In-Memory- Linux, CentOS, Ubuntu- Java, Scala, Python- Spring, AOP, Multi-Thread- HTML5, Javascript, Sencha, ExtJS- PostgreSQL, H2Database

http://www.cloudine.io