Oleg Miagkov Email & Phone Number

Москва, Россия

Московская Область, Россия

Russia

Project - Data Platform.Project tasks:- Development and support of DWH.- Creation and maintenance of a statistics collection system with 40+ million events per day.- Ensuring reliability and scalability of highly loaded services.- Development of infrastructure based on Hadoop and GreenPlum for loading and analyzing data.- Preparation of data for machine learning and development of web services for integration with products.- Creation of Data Catalog.- Integration… Show more Project - Data Platform.Project tasks:- Development and support of DWH.- Creation and maintenance of a statistics collection system with 40+ million events per day.- Ensuring reliability and scalability of highly loaded services.- Development of infrastructure based on Hadoop and GreenPlum for loading and analyzing data.- Preparation of data for machine learning and development of web services for integration with products.- Creation of Data Catalog.- Integration of products on the platform.Technologies: Python, Golang, SQL, Apache Spark, Apache Kafka, Apache Airflow, Redis, Greenplum, Postgres, Hadoop, Kubernetes, Datahub, Great Expectations Show less

Москва, Россия

Customer - The largest sportswear manufacturer in Europe, and the second largest in the world.Project - Payments analyticsRole - development team leaderResponsibility :- Managing the development team- Task planning- Interaction with the customer- Collection of raw data in file storage- Batch data processing- Stream data processing- Building a data lake- Uploading data to the data warhouse- Building data martsTechnologies: Python… Show more Customer - The largest sportswear manufacturer in Europe, and the second largest in the world.Project - Payments analyticsRole - development team leaderResponsibility :- Managing the development team- Task planning- Interaction with the customer- Collection of raw data in file storage- Batch data processing- Stream data processing- Building a data lake- Uploading data to the data warhouse- Building data martsTechnologies: Python, Apache Spark, Apache Kafka, Kafka Connetors, SQL, AWS: EMR, Hive, S3; Jenkins, Kubernetes, Exasol Show less

Саратов, Саратовская Область, Россия

One of the top 3 leading universal banks of Russia. Bank and its subsidiaries form a leading Russian financial group, offering a wide range of banking services and products in Russia, CIS, Europe, Asia, Africa, and the U.S.Data delivery to the enterprise data lake from internal and external sources.Organizing the work of developers, planning the backlog of tasks, designing data flows for appends in data lake from internal sources( databases, file sources) and external sources (… Show more One of the top 3 leading universal banks of Russia. Bank and its subsidiaries form a leading Russian financial group, offering a wide range of banking services and products in Russia, CIS, Europe, Asia, Africa, and the U.S.Data delivery to the enterprise data lake from internal and external sources.Organizing the work of developers, planning the backlog of tasks, designing data flows for appends in data lake from internal sources( databases, file sources) and external sources ( public ftp, parsing sites), replication data. Optimization of Sql queries.- Managing the development team- Batch data processing- Interaction with the customer - Collecting raw data to file storage- Development ETL pipeline- Batch data processing- Development SQL queries to perform data extraction from existing sources in order to check format accuracy- Imported, transformed and set in the right format- Was in charge of the user stories refinement, tasks planning and estimations format accuracy Show less

Russia

IoT platform for the automobile manufacturer- Building data-lake for storing data from an IoT devices and infrastructure.- Batch data processing and building analytic data martsTechnologies: Cloudera DP, Apache Hadoop, Apache Spark, Impala, CloudantDb\CouchDB, PostgreSQL, ParquetBuilding a real-time analytic platform based on Lightbend Cloudflow.io (Apache Spark, Apache Flink). Construction of a monitoring system (triggers) of business events of the Credit Bureau… Show more IoT platform for the automobile manufacturer- Building data-lake for storing data from an IoT devices and infrastructure.- Batch data processing and building analytic data martsTechnologies: Cloudera DP, Apache Hadoop, Apache Spark, Impala, CloudantDb\CouchDB, PostgreSQL, ParquetBuilding a real-time analytic platform based on Lightbend Cloudflow.io (Apache Spark, Apache Flink). Construction of a monitoring system (triggers) of business events of the Credit Bureau, aggregaton raw-data, building data-marts.- Service architecture development- Business logic development- Implementation of streaming pipelines on Flink (stateful / stateless)- Collecting raw data into file storage (spark streaming, kafka, hdfs and delta lake)- Batch data processing and building data marts (spark sql)- Services Flink (cloudflow.io) and Spark 3 in kubernetes (rh openshift) Scala, Akka, Akka-Streams, Kafka, OpenShift, Cloudflow.io, Apache Flink, Apache Spark, Delta Lake Show less

Saratov Region, Russian Federation

Building Enterprise Data WarehouseResponsibilities: -Development pipeline for migration data to cloud -Development ETL pipeline -Building Enterprise Data WareHouse -Developed SQL queries to perform data extraction from existing sources in order to check format accuracy -Imported, transformed and set in the right format -Database migrations from Traditional Data Warehouses to Cloud -Worked with the team to deliver components using agile software… Show more Building Enterprise Data WarehouseResponsibilities: -Development pipeline for migration data to cloud -Development ETL pipeline -Building Enterprise Data WareHouse -Developed SQL queries to perform data extraction from existing sources in order to check format accuracy -Imported, transformed and set in the right format -Database migrations from Traditional Data Warehouses to Cloud -Worked with the team to deliver components using agile software development principles.Achievements: -Successful migration data to Google Cloud platform from different sources(MSSQL, ORACLE, DB2, MySQL, FILE SOURCES) -Successful implement ETL pipeline in cloud platformTechnologies: Python, Scala, SQL, Google Cloud Platform: Big Qeury, DataProc; AirFlow, Spark Show less

Saratov Region, Russian Federation

Enterprise solutions for bank systems, microservice architecture, OpenAPI. Technologies and products:Java 8, Spring Framework (SpringBoot, JDBC, JPA, MVC), Red Hat OpenShift, Mysql, Oracle DB, Aerospike, Tarantool, Gradle, Maven, GitLab, Jenkins, Docker, WSO2 APIM/IS, IBM DataPower, IBM APIC, Kafka, Kafka Streams, Avro, gRPC, Protocol Buffer

Saratov Region, Russian Federation

Enterprise solutions for bank systems, microservice architecture, OpenAPI. Technologies and products:Java 8, Spring Framework (SpringBoot, JDBC, JPA, MVC), Red Hat OpenShift, Mysql, Oracle DB, MongoDB, Eventuate.io, Gradle, Maven, GitLab, Jenkins, WSO2 APIM/IS, IBM DataPower