Vítor Castro Email and Phone Number

Tech Stack:- Python 3.12.- Docker CLI to simulate some AWS services and test locally.- Amazon S3, Amazon SNS, Amazon CloudWatch, AWS IAM, Amazon VPC.- AWS Glue for ETL.- AWS Lambda and Spark/PySpark for data processing.- AWS Step Functions to orchestrate the data processing pipeline.- Amazon Athena to query the data.- AWS Lake Formation for data governance and access control.- QLIK for BI and data visualization.- (VS Code, Microsoft SSMS, Azure DevOps, Asana, Postman...)

Responsible for developing new features for Metric, a software tool that supports telecommunication companies in network planning, monitoring, and optimization. This role involves addressing and resolving complex bugs to ensure the seamless operation of the platform. Metric handles an influx of 1 to 2 TB of data per month, encompassing the configuration and performance metrics of all antennas across the entire network. This data is critical for making informed decisions and optimizing network performance.Tech Stack:- MySQL used in the back-end.- AWS (Lambda, S3, SQS, etc.), and Terraform for data processing.- AWS Glue for ETL.- Serverless Framework to develop and deploy Lambda functions.- Python 3.9 (pandas, scikit-learn, dowhy, plotly, pyarrow, matplotlib, numpy, etc.) for building the ML pipeline.- unittest for Python back-end testing.- AWS (Athena, SageMaker, QuickSight) to train, build, and deploy ML models.- Trino (PrestoDB).- Parquet file format.- PySpark for data processing.- Flask.- KNN.- (DevTools, Git, Bitbucket, Redmine, JetBrains, Jupyter Notebook, Json, XML, Bash...)Main Achievement:Implemented a data storage optimization strategy that transitioned from storing TSV format files on Amazon S3 to utilizing PARQUET format files, resulting in a significant reduction of approximately 90% in monthly storage costs. Successfully managed the transformation of over 1TB of data per month, demonstrating effective problem-solving and cost-saving abilities in data engineering across the entire data processing pipeline.

Responsible for developing new features for Metric as well as fixing uncanny bugs. Metric (https://metric.pt) is a software tool to support telecommunication companies in network planning, monitoring, and optimization.Tech Stack:- Javascript with AngularJS, LESS/CSS, HTML, TypeScript, and CoffeeScript used in the front-end.- PHP 8.2 (with Slim) and MySQL used in the back-end.- C# (.net6.0 framework), AWS (Lambda, S3, SQS, Glue, etc.), and Terraform for data processing.- JRuby and Bash for development and deployment.- Karma and Jasmine for front-end testing.- Codeception for PHP back-end testing.- xUnit for C# back-end testing. - AWK to manipulate .tsv files.- Serverless Framework to develop and deploy Lambda functions. - (DevTools, Git, Bitbucket, Redmine, JetBrains, Jupyter Notebook, Bash...)

Processing data from one of Portugal's largest energy companies.Fine-tuning pipelines and optimizing database queries to ensure smooth monthly billing for clients, as well as troubleshooting billing errors.Tech Stack:- Java for building pipelines.- Python for scripting.- GCP (BigQuery, Dataflow, Bigtable, etc.), and SQL for data processing.- Excel to gather data insights. - (Git, Bitbucket, Jira, JetBrains...)