How to Evaluate Trade Off in Spike
The hardest thing about technical documentation is not about the technical aspect. It is how to persuade your audience to choose a certain solution.
One of the hardest parts about designing a spike is not coming up with the solution; it is how to evaluate the best solution. However, I often found it hard to evaluate the best solution because each stakeholder has opinions and needs.
How do you devise the best solution for the problem and pitch that choice to your technical and nontechnical colleagues?
Every stakeholder has a different opinion. I cannot accommodate all of them.
You have investigated and written a technical design proposal and laid out each pros and cons of the solution in bullet points. You have proclaimed out of solutions A, B, and C; you will choose solution A because it has the most pros and the least cons.
You invite all the technical and non-technical project stakeholders to discuss the proposal. During the discussion, you describe why you chose solution A because of the pros and cons compared to B and C. As you favored solution A, you noticed that the stakeholder's atmosphere doesn't think otherwise.
John from the security team said solution A is unreliable and doesn't abide by certain security rules. Judy, the product manager, said that she favors solution B because it decreases the time to market. On the other hand, Asher from the engineering team said that solution A didn't cover a couple of disadvantages that will create technical debt to their service code base, so he proposed adopting solution C, which requires a migration work prerequisite before working on the project. Meanwhile, PRD (product requirement description) mentioned that this feature has a deadline of one week from the current presentation date.
Who is right? Everyone is right to accommodate these solutions based on the requirements of their team. How could you prepare your documentation better to evaluate a particular characteristic vs. another?
The perfect system is the system that can accommodate that feature/project.
Context Matters
The problem with the example above is that we construct our pros and cons based on assumptions. We have to have benchmark criteria to evaluate our technical solutions and have alignment with all stakeholders.
Technical decisions depend on the context of the business. Understanding the project's most important aspect and each stakeholder's motivation helps you capture and describe the solution more effectively. I am often stuck in writing pros and cons because I don't have a benchmark against what matters most to the project.
For example, reliability and scalability are often stressed during system design interviews. Thus, many software engineers value reliability and scalability in every aspect of their technical design documentation. Making the feature reliable and scalable creates a superb user experience when you have a product-market fit. However, when you are prototyping the feature, reliability, and scalability is one thing that will slow you down! In this scenario, you must design your system considering the short time to market duration.
Understanding the goal of the project of your team and organization is paramount to brainstorming the right solution. For instance, how important is the user experience in the app? Does the interaction need to be in real-time? Shall we maximize independence and not reliant on other teams for future changes? How often will the system be changing? Do we want to minimize the maintenance effort from another team? How fast do we want to build this feature? When is the deadline? Let's remove the deprecated code and not increase more things on the debt pile first.
Aside from understanding the business context, we need to understand the needs of each stakeholder. You need to gather information and ask these questions before designing your solution.
From these questions, you can capture the most important criteria from your stakeholders and highlight them in bold with High and low score criteria. Then, you can discuss with your stakeholders beforehand what you will be comparing your solution against so everyone is on the same page.
Here are a couple of things that I think about the pros and cons of a certain design solution:
User Experience: How does the solution impact the user experience regarding up-time/latency/consistency?
Implementation Score: How long does it take to implement the solution?
Maintenance Score: How easy is it to add a new feature? In other words, how many days will it take to extend a new feature? Can it be self-serve, or does it require an expert resource?
Clean-Up Score: Does the proposed solution enable easy clean-up if we don't need it anymore, or once we did the solution, it has limited ability for code removal?
Once you have an alignment on what the ideal solution looks like, you can start working on the technical solution.
Design A Working Solution
At this stage of your technical design phase, write down one potential solution with the least effort, regardless of the important criteria. The reason why you want to design the solution with the least amount of effort is that you don't want to overwork yourself or your team. The solution may be super hacky, but that is okay. The goal in this phase is to design the system that works.
For instance, if you are told to design a notification opt-in solution, you can design one solution. The solution is to reuse the current notification preference database and add the opt-in-enabled field. Then, you can draw the high-level user diagram.
Incorporate Pros and Cons
Once you have a working solution, it's time to capture the pros and cons of the criteria design criteria.
Think in Terms of User Experience.
How will reusing the notification preference database impact the user experience regarding up-time/latency/consistency? How many more requests will that database need to handle, and will that impact the existing operation? What will be the access pattern of the user? Since opt-in operation is more a write-heavy operation than a read-heavy operation. How will that tied to the existing preference database where a read operation is always called when the user goes to their settings page?
Think in Terms of Implementation Score
What is the level of effort of such an approach? How many engineering resources do we need to build this feature? Is there any dependency on other teams on certain processes to be completed to finish the feature?
Think in Terms of Maintenance Score
How much effort does it require If we want users to be able to opt-in for multiple notification criteria? What happens if they have problems with their opt-in process? Is it a self-serve process to troubleshoot, or do we need expert resources?
Think in Terms of Clean Up Score
If, for some reason, the product decided to forgo the opt-in feature and focus on something else. How easy it will be to deprecate the flow? How will that impact the current existing database?
Think about the cost and benefit of these tenets.
You can compare these answers to the important criteria and iterate on your solution. The answer to these questions will be the pros and cons of the approach, and you can create another option for the alternative approach. Then, you can rinse and repeat the steps above with the other alternative solution by asking about these four aspects.
Couple Tips on Using This Method
Choosing between "hack" vs. the "right way."
A common tradeoff is development time. Should we opt for a "hackier" solution and do it "properly" later? A fast and "hackier" approach allows us to get feedback faster but will hurt user experience (i.e., reliability, scalability, and developer experience). Suppose you find that the product and the team always favor fast development time and "hackier" approach but realize that technical debt has bitten your team developer time and hurt user experience. In that case, you must address that by mentioning the cost of tech debt.
If you want to describe the cost of tech debt, describe in a way how long and how many resources are needed for maintenance. Illustrating the impact of incurring tech debt helps the product visualize the cost of that technical debt.
Keep Your Option to a Three
Making too many alternative solutions requires a longer research time and makes it hard to explain those decisions to the stakeholders. Start with your main approach, and map that approach with the criteria that you gather. If you have too many options, start cutting them down by evaluating them based on the criteria that matter and are "strictly better." Having three options helps keep the solution focused on the criteria and help your stakeholder navigate their decisions.
Conclusion
As engineers, it's easy to think about the tech stack, develop an intuition, and make tradeoffs quickly. But we aren't aware of the business model we are operating under. We either get frustrated or don't understand how management makes decisions.
To make good decisions, you not only have to understand the cost, you also have to understand the benefit. Mostly, those are driven by the product context. Before designing your solution, you should align all the criteria that matter to your stakeholders and the feature.
Once you align, you can start with a working solution, map out all the pros and cons based on the criteria, and branch out to the alternative solution.
Last, keep your solution to three to not overwhelm your stakeholders and create decision paralysis.
Designing technical solutions with the right tradeoff requires constant practice and communication with repetition and with the help of others for a working solution.