Pavel Olifer Email & Phone Number

Saint Petersburg, Ru

Moscow, Russia

1. Migrate pipelines from Airflow v2.1.2 to v.2.5.32. Automate refresh excel reports3. Participate in the framework development for Excel refreshing (Python)4. Refactoring Airflow pipelines to optimize performance5. Refactoring / Formatting Python code

Saint Petersburg, Russia

1. DWH DESIGN- DWH architecture (OLAP vs. OLTP, DWH Layers, Data Lake, Data-Mesh, Inmon, Kimball)- Normal forms- DIMENSIONAL modeling methodology (SCD, Star, Snowflake, Constellation)- DATA VAULT methodology (Data Vault 1.0, 2.0, Recreating Dimension table and Fact table from DV)- ANCHOR modeling (AM) methodology (Anchor, Link, Attribute, Node, AM & 3NF)2. RELATIONAL AND MPP DBMS- Architecture of Relational and MPP databases (PostgreSQL, GreenPlum)3. ETL PROCESS AUTOMATION- Getting to know Airflow- Complex pipelines- Developing your own Plugins- Installing and configuring Airflow4. BIG DATA - Hadoop basics. HDFS, YARN, MapReduce- Hive. HBase- Spark basics- Kafkа. Spark streaming- Debugging, profiling, and monitoring Spark job5. CLOUD STORAGE- Introduction to Clouds (IaaS, Paas, SaaS)- Cloud infrastructure- Kubernetes for Engineering- Deploying Spark in Kubernetes6. DATA VISUALIZATION- Tableau basics- DataLens and SuperSet review7. BIG ML- Introduction to Machine Learning - Distributed Machine Learning theory (Model lifecycle, Model complexity, Transfer learning, Distributed ML, Frameworks, Spark & ​​GPU)- SPARK ML (Transformer, Estimator, Evaluator, Pipeline, Embedding, Hyperparameter tuning, Supported model classes)- Spark ML. practice (Data preparation, LogisticRegression, DecisionTreeClassifier, RandomForestClassifier, Gradient-boosted tree classifier, Save & Load Model, Pipeline, Hyperparameter tuning)- Applying undistributed ML Models to Big Data (Spark UDF, Pandas UDF, Sklearn model)- Project: Bot identification8. MODEL MANAGEMENT- ML model training pipeline (ML Lifecycle, Versioning, Feature store, ML Pipeline Management)- Data Versioning (DVC, AWS CLI)- Using MLFlow for Model Tracking and Accounting (Introduction to ML Flow, Experiment management and tracking, Application and serving of models)9. DATA MANAGEMENT- Data management- Data Security- Data Quality- Hands-on with Deequ for Data Quality

Moscow, Russia

Largest vertically integrated infrastructural company, combining export logistics and grain trading assets.1. upload data from API (MOEX, SPIMEX, Yahoo)2. upload data from big excel files (~ 250 Mb)Main stack: Python, Rest API, Pandas, PostgreSQL, Gitlab CI/CD, Git, Prefect

Moscow, Russia

Working as consultant for private invest fund.0. All interaction takes place in English- within team- with others team- oral (zoom meating), written (email, slack)1. Migrating data: - Oracle to S3 to SnowFlake (~ Pb) - Snowflake to Oracle (partitioning tables in Oracle) - HDFS to S3 to Snowflake2. Creating pipelines in Apache Airflow (json)3. Migrate PL/SQL code from Oracle to Snowflake4. Scripting on python: - create pipelines & source sql (all dml operations: delete, insert into, select) - to get informations for Oracle, Snowflake objects - to get row count for table list (~ 157 tables) - create HUGE materialized view ( ~ 1.5 Tb) - check columns between Oracle & Snowflake, record result into excel5. Using Gitlab (git) for deploy ready pipelines in DEV / BETA / PROD env.6. DBT, FlyWay7. Data QualityMain stack:- Apache AirFlow- SQL, Python 3- Oracle, SnowFlake- AWS, Docker, Linux- Gitlab, Git, Jira, ConfluenceAdditional:- Django- weChat API- telegramBot (aiogram)

Saint Petersburg, Russian Federation

- maintaining the operability of 3 ERP-systems and services associated with them- completion of the existing functionality of the ERP system (warehouse logistics)- development of new functionality of the ERP system- interaction with third-party foreign developers (mail, chats - correspondence in English)- creation of database objects in Oracle DBMS- work involves roles: Fullstack developer, System analyststack:- PL/SQL, SQL, XHTML, JavaScript, VB, Java(*.jsp), Python3,- Oracle 11g,- Win, Linux (Ubuntu, ),achievements:- created console-application which listening range of ports, take and insert data from/into Oracle DB (Windows, Oracle 11G, Python (2.7, 3), Socket, Threading;)- development of a web service for printing Excel files(Windows, Oracle 11G, Python (3), Flask, Openpyxl, win32API)

Saint-Petersburg

- Maintenance of the existing functionality of an ERP-system (Salary and Personnel)- Development of a new functional ERP-system- Participation as a developer / system analyst / business analyst in the ERP-system integration with other systems (Active Directory, Axapta, 1C, Tableu, Documentum, Systems of Working Hours)- Creation of database objects (packages, procedures, functions, triggers, types, collections, jobs, views, materialized views, temporary tables)- Optimization of the reports execution time (the goal was to decrease the time of report creation from 8 hours to 40 minutes)- Process automation (printing documents related to hiring of a new personnel)Achievements outside of work (personal development):1.Creating a parser for a russian job posting site.The bottom line of the parsing was to find all St. Petersburg job descriptions where food compensation was mentioned.The API site was used.Platform: Python (2.7) + Flask2.Participation in the project as a freelance designer to create the forms and logic to the judicial system.During this project Pavel became acquainted with such programs as Oracle ADF, Tortoise HG, Maven, Jira.I had to install a virtual machine VMWare + Oracle 12c and configured my Mac PC to work with the database of the SQL Developer.Thanks to the project, I realized that I needed 2-3 months to study an unfamiliar IDE and create a system with minimal functionality. It was my first freelancing experience.

Moscow, Russian Federation

Senior IT SpecialistWas promoted to Senior It Specialist and kept old duties as well as got new responsibilities such as:- maintenance of the existing functionality of an ERP-system (calculating the cost of transportation services for Hyundai Motors corporation) upon the request of department heads- development of a new functional ERP-system to store orders for spare automobile parts used languages/technologies/tools:- JS (* .aspx), C # (* .aspx.cs), Java (* jsp.), PL / SQL, SQL- Oracle 10g, 11g, 12c, Weblogic, IIS, .NET 2.0- TrustForm IDE- Toad- Visual StudioAchievements:Preparation of the equipment for the ERP-system:- Installation of a DBMS (Oracle 11g)- Installation of the software required to operate- Web server tuning (Weblogic, IIS)Creation of the ERP-system calculating spare parts transportation scheme from warehouse to warehouse, from storage to dealer, from dealer to warehouse- Development of the database structure- Creation of database objects- Programming and development of algorithms, according to business logic processes- Automation system for a person of minimum intervention- Development of web-services (C #)- Reporting from/to excel

Saint Petersburg, Russian Federation

IT Specialist- Maintenance of the existing functionality of an ERP-system (warehouse logistics, calculating parts availability in the warehouses) upon the request of department heads- Development of new functional ERP-system- Support and training of the ERP-system users- Interaction with the heads of departments and ERP-systems developers- ERP-system administration (access rights, server administration, application server)- The creation of database objects (package, procedure, function, trigger, index)- Optimization of SQL-queries- Tuning equipment (printers, thermal printers, PDA)- Keeping tasks in Redmineused languages/technologies/tools:- HTML / XHTML, JS (jQuery), Visual Basic (* .asp), Java (* jsp.), PL / SQL, SQL- Oracle 10, 11, Weblogic- TrustForm IDE- Toad- Visual StudioAchievements:- Implementation of a new business process to plan the unloading of parts on the basis of priorities;- Fixing critical mistake in the business logic blocking large amounts of spare parts from being sent to the car assembly line.

Saint Petersburg, Russian Federation

- Work with Oralce 9i, 10g;- Creation of database objects: procedures, triggers, functions, etc. (PL/SQL)- Writing queries to the database to provide data users (sql);- Create, modify forms - Oracle Forms 6;- Create, modify reports - Oracle Reports 6;

Saint Petersburg, Russian Federation

ERP-система для вуза (договора на обучения, планы обучения, оценки по предметам, информация по студентам, абитуриентам и т.д.)- работа с Oracle 10g, 11g;- написание пакетов, процедур, функций PL/SQL;- написание запросов SQL, оптимизация кода;- разработка, исправление, изменение отчетов - Oracle Reports 6,9;- разработка, исправление, изменение форм - Oracle Forms 6;- написание ТЗ, инструкций к разрабатываемому функционалу;- дополнение и расширение функционала "клиента";- консультирование и помощь в работе с "клиентом";- datafix - выявление, исправление некорректных данных;достижения:- разработка базы данных, процедур, функций для загрузки классификатора адресов (КЛАДР) в рабочую базу, с возможностью обновление его на новую версию.

Mathematician Programmer, Mathematical Support And Administration Of Information Systems, Retraining

Master'S Degree, Laser And Optical Engineering