Senior Software Engineer
CurrentI was hired by the VP of engineering. I’m working as an individual contributor in data infrastructure and backend projects.- REST API maintenance in blockchain’s domain endpoints while decommissioning some parts of legacy code and developing new others;- I contributed to the blockchain’s pipeline development not only by adding new pipelines or features but also by adding automated tests, decommissioning, and refactoring code to reduce technical debt and increase… Show more I was hired by the VP of engineering. I’m working as an individual contributor in data infrastructure and backend projects.- REST API maintenance in blockchain’s domain endpoints while decommissioning some parts of legacy code and developing new others;- I contributed to the blockchain’s pipeline development not only by adding new pipelines or features but also by adding automated tests, decommissioning, and refactoring code to reduce technical debt and increase maintainability;- I built data normalization pipelines for chain data;- I rebuilt crawlers to fetch blockchain data across the web for multiple chains, with means to fix outstanding issues while increasing performance and maintainability. Additional features like proxying + exponential backoff were implemented to avoid blocklisting;- I built ETL programs to move data from structured databases to object storage with minimal resource usage, while inserting GBs worth of data in minutes;- EVM data pipelines implementation using Substreams with Rust and Go, consuming and storing TBs worth of data in a matter of hours while keeping the resource usage to a minimum;- I customized open souce software to fit the needs of our company, such as creating forks to add new features or fix bugs that don’t need upstream intervention. Notable examples are: Ethereum ABI decoding project to support lower Node versions, and Substreams file sink project to support binary file writing in Apache Parquet format and proper upload to cloud object storage;Technologies and concepts used:Programming languages: Go, Rust, Scala, Python, and JS/TS;Storage: PostgreSQL/TimescaleDB, Apache Pinot, and S3;Containerization: Docker, and k8s (AKA Kubernetes);Data Engineering: Apache Airflow, Apache Kafka/Redpanda, Kafka Streams, Apache Parquet, Kafka Connect, and Databricks;Ops-related goodies: AWS CloudWatch, Datadog, Grafana, Prometheus, Argo CD/Workflows, and GitHub Actions;Cloud: AWS. Show less