- scala services (data platform)- spark streaming- spark ETLs (+ luigi, airflow)- infra (k8s, helm)

- Spark@scala, airflow, k8s for robust and monitored ETLs- developed internal data labelling tools (python+js) 5X labelling speed boost- found data bugs and wrote DQ rules- started to integrate a data catalog- made some helper services like Telegram Bots

Supported the business operations by integrating WMS and the necessary glue.

- worked with SQL and NoSQL databases- created an end-to-end storage for country-wide data using MapReduce and ElasticSearch- brought Python packaging practices and infrastructure (PyPi, Jenkins)- done data visualisations- worked with teammates to monitor Hadoop cluster and Jobs- done ad-hoc analytics and parsing using Python, tested ML models manually- implemented the Puppet configs and Zabbix triggers- gone through very best code review and had cool mentorsResponsibilities:- business reporting in graphical and tabular way- ML models data supplyStack:- MapReduce Streaming- Postgres, ElasticSearch- Python, pandas, seaborn- PyPi, Puppet, Jenkins