Responsible for the discipline of Systems Performance, Cloud, Infrastructure, tools that fit with all these areas and, definitely not least important, responsible for presales of everything that fits with my area. Development, watching and guiding the whole team are daily activities, besides being the quality keeper of all work done by the team to our customers. We are a high performance team, so automation is our breakfast. Besides we have a strong knowledge in Performance by Design, Performance Tests, Observability, IaC, Configuration Management and Multi Cloud. Focused on understanding our customer necessities and issues in order to create a solution driven by excellence.

My first week at First Data was really chaotic. I started in the middle of a crisis and I was dragged to a war room as soon as I picked up my notebook. It took the whole day to figure out what was happening, but I realize there were only 2 instances of the application and each instance with 60 Gb of Heap. So, everytime a full gc occurred, a huge application freeze was felt by the customers. With a simple split of 2 instances to 8 instances, decreasing these huge heaps to 6 Gb each and a few adjustments on the load balancer was enough to bring stability and performance to this system.At my first weeks of on-call, I realized the majority of calls were about disk reaching their capacity, false alerts or in same cases lack of monitoring, lack of business understanding and lack of sanity check routines that could be run for a junior analyst.I began a long way to automate actions like compress, archive and clean old logs. Create a few sanity checks scripts for junior analysts in order to do a further big picture that wasn't given by the actual monitoring. These scripts were really important to tuning our monitoring (SLIs and SLOs) and avoid false alerts or, what is the worst, false positives. For each new system that was on schedule to be delivered, I was involved to understand the criticality, which SLIs we are going to monitor, what are their SLOs and what is going to happen if these SLOs stay under the threshold.Once in a while, a crisis happens. After all the troubleshooting finished and the problem got solved. A postmortem was done in order to understand what happened, why it happened and what should be done to avoid it from happening again.

During my short time at Rivendel I learned a lot about AWS Cloud. The project that I worked on was a data lake creation for a Financial Institution. As a matter of fact I didn’t work only at that creation. I worked on the encryption process of sensitive data that is leaving the DC towards the cloud. Since all data was sent over the internet, the best option was an asymmetric cryptography (RSA) for transport and then it’ll be decrypted at cloud and changed to a symmetric cryptography. A two phase process was created to deal with this encryption. Since data was acquired, they were sent to a pipeline on Apache NiFi that was used to enrich or, in this case, secure the data. The cryptography job was built on top of java and used on NiFi as a custom node process. As data arrives at the cloud, another NiFi was in charge of decrypting the data with a private key and encrypt it again with an AES key.Since I know in advance that a lot of tasks will be done repeatedly, everything that involves setup I did using Ansible to avoid doing it manually once again.

I started to lead a small team of 5. Teaching them was something that I really enjoyed. I started to create a few training and challenges for them in order to improve their knowledge quickly. I also start to visit our customers as a pre-sales. The idea of seeking to understand the others' necessity, what they really need, made me fulfilled. There are some cases when the customer is looking to the wrong side of the problem and trying to solve the symptom not the root cause. Helping them to look in the right direction is really grateful.

As a senior engineer I began to handle complex issues like solving undocumented framework bugs or bugs made by software developers that forgotten to use appropriatly design patterns.One of the most pleasant projects was a stress test on an acquiring company. We have to simulate a POS sending debit and credit transactions using ISO 8583 protocol. The big deal was there wasn't any free tool for this at that time. So we built our own. Using java and a lot of LinkedBlockingDeque to ensure the asynchronism, we could achieve a maximum of 10.000 TPS with a standalone jar application using 4 cores and 4 Gb of Heap.

As I gained experience I began to deal with more complex tasks like debugging and troubleshooting slow systems. When it was a product of a performance test, we used to have a bunch of tools for monitoring and this made the analysis really quick. But once in a while I was called for troubleshoots on customer systems and they don’t have efficient monitoring really often or in a few cases have no monitoring at all. In this case I have worked with native tools like jstack, mission control, heap dumps, thread dumps, in java applications.A deep understanding of JVM and other protocols like HTTP and TCP was really necessary. Having knowledge of a slow query could rapidly drain your entire connection pool and thread pool, for instance, makes all the difference.Instead of only doing stress tests to seek the maximum of what a system could handle, in a production environment, we used to do a capacity planning study. It consists of the correlation of a business metric like amount of online users and the server metric like CPU usage. With these two metrics a statistical study were made and a model was created to predict how many cpu will be needed with the number of customers online increase in 50%, just for instance

In InMetrics, I started to work directly with system performance, dealing with kernel tuning to avoid SYN Flood messages and dropped packages during a high system usage, and testing exhaustively the performance of a huge number of systems, with distinct technologies and architectures.Before the test planning began, a critical talk always had to be done: The Monitoring. To start this task we must have a deep understanding of the system design, the architecture tradeoffs, all integrations that will be stimulated during tests. All this knowledge is a must have for planning a precise observability, including application logs, server and system metrics and traces. Although all these data were available we always had one or two meetings with customers to define their SLO. In a few cases we had to argue with them because, for instance, a 100% of response time under 200 ms wasn't a realistic measure.Since performance tests were my major work, I started to create a few automations to avoid repeated work like generating charts for a report. As a rule, I used to spend a day doing all the charts for an overall result, but with a python script with pandas lib, I began to finish this job in a matter of 1 hour.

As a Junior analyst I started handling simple tasks like running automated scripts to setup a server or answering the phone when customers had something to complain about their systems. As the time came by, I began to deal with more complex subjects like system deployments specifications, network troubleshoots, firewall situations and linux kernel tuning.In my last 5 months I created a bash script to run a health check on a server and give me some insights like there are processes using swap area and these are the PIDs that are using it.