- Data source analysis(Oracle, PostgreSQL, MSSQL, Kafka), configuring the data collection process, and loading into data lake(Hadoop); monitoring data quality (completeness, alignment with business metrics, SLA).- Analyzing query performance, configuring allocated resources(configuring Spark parameters), optimizing table structures, utilizing indexing mechanisms, data segmentation, and partitioning.- Oversight of ELT pipeline initiation and completion(AirFlow), including data integrity checks. Handling notifications regarding issues in the data storage and transformation tools. Resolving issues with related departments in data storage and ELT tools.- Designing data marts: analyzing business requirements, designing the necessary table systems, integrating into storage for future reuse.- Conducting code reviews, ensuring error-free and optimal code performance, providing consultation and assistance to colleagues in code productivity, ensuring compliance with code formatting agreements

- Installation, configuration and support of solutions for processing big data at the customer's site (Spark + company's own solutions)- Setting up ETL based on NiFi, writing flow, tuning parameters - Using SQL to fetch data from client databases and load them into the data lake- Setting up orchestration with Air Flow- Setting up distributed file systems (Gluster/HDFS)- Deployment automation (Ansible + Python SDK) of machines with the company's software solution to the clouds (Oracle cloud, Yandex Cloud)- Writing scripts and additional utilities (gathering diagnostics) in Python- Trying not to break git

- Administration of Splunk installations for customers (both Splunk itself and the Linux system environment) - Assistance to customers in writing analytical queries on SPL(Splunk) - Setting up data source connection in Splunk(ETL)- Administration of Hadoop installations for customers (Cloudera) - Solving problems with the operation of HDFS, HIVE, Impala, Spark and other Cloudera services- Writing documentation for customers

- Commissioning and administration (backup, logging, load balancing) of iron and virtual (VmWare) servers (CentOS, Debian)- Deployment and administration of the company network- Administration of monitoring tools for servers and network equipment (Zabbix, Nagios)- Using of a configuration management and containerization system- Writing and adaptation of scripts for automation (bash + python)- Commissioning and administration of VPN servers to connect remote offices and employees (OpenVPN, L2TP/ipsec)- Use of cloud solutions Amazon, DigitalOcean- Maintaining documentation (internal Wiki)