[Project D] Devlog #2: Reporting
Before we begin, if you want to know where we started, kindly see the first post in this series. With that out of the way, let's get started.
[Project D] Devlog #1: Debugging a failing request
I didn't do much in the way of coding today, I spent quite a bit of time thinking about how to do the implementation for some reporting that needed to be done.
The idea for the reporting service for my microservices is to make it such that the data that is needed for the reports is saved elsewhere so that when we're querying from the frontend, we do not have to run the aggregation at that time. If we can make it such that we're just doing select queries, that should in theory reduce the time that is needed to get the data.
For instance, if the query is as simple as select reporting data from reporting_table it would be much faster than if we had to do something involving a whole lot of joins. (The system is very interconnected, so there will be a lot of joins in certain places).
In execution, it was not as straightforward. First of all, I needed the make the following decisions
At what point do I save the data into the "elsewhere"?
Do I do this when I am saving the new data? (what about old data?)
Do I schedule a task to run at a specific time to move the data for a specific day to "elsewhere"? (How does this affect the system usage at that time?)
What does the data look like "elsewhere"?
Am I creating a new database for the reports with separate tables for each entity that needed to be recorded?
And so on and so forth. My mind was literally filled with so many considerations that I could not make a decision. It then occurred to me that I had fallen into the pit of premature optimization. I mean, I hadn't even written a single line of code or modeled my data for what the responses would look like.
In light of this, I decided to go with the naive approach. Get something working first, understand the data, and then optimize it. So that is what I did. I used JPA to generate a few methods for counting some entities and wrote a few queries using JPQL to help me understand the data.
That is where I'm currently sitting, I would have loved to work on the payment today but my webhook still hasn't been activated yet so there's not much I can do about that.
I will continue to build out the reports and then see how best I can optimize it in the coming days.
Until next time, I bid you adieu.