To Be a High Performing Software Engineer, Adopt this Thinking Framework to Solve Complex Problems
Have you heard of First Principle thinking?
My friends and family who do not work in tech don't understand why software engineers get paid so much.
"Coding tutorials are everywhere nowadays, and for most applications you want to build, you can search in Stack Overflow. You can do the job if you know the programming language. Why are software engineers paid so much?" asked my friends and family.
Software engineers get paid a lot because they often need to turn abstract ideas into executable steps and solutions.
Some of the problems they encounter include:
The company contains multiple product lines in different pipelines and wants to enable bundling. Engineers need to figure out how to create such solutions.
3DS is an authentication protocol in payments for businesses to verify user credit cards with the bank before authorizing the users’ payments. Currently, many different payment gateway solutions provide such solutions for a merchant to integrate 3DS. If merchant A integrates with payment gateway B, merchant A must use the integration method supported by gateway B. Thus, if merchant A wants to integrate with payment gateway C, they will need to adopt a different integration method from gateway B. Businesses want to use multiple gateways to initiate the 3DS protocol to increase their authorization rate. How do we create a multi-tenant strategy in the 3DS payment processing system? Note: move this one last; too long
Business wants to create better engagement by targeting marketing campaign to their user base. They ask their engineers about how to do such a targeting marketing campaign. They also ask if those ideas are feasible in this time frame.
The company wants to track the impact and causality on a campaign funnel. Management asks software engineers to understand if such an idea is feasible.
The best engineers I have worked with can create an execution plan from abstract ideas, and most of them adopt the First Principle thinking framework to solve complex problems.
First Principle thinking is the most effective strategy you can employ for breaking down complex problems and generating original solutions. Many engineers adopting this framework can develop a solution that is innovative and outside the box.
"First Principle thinking is a fancy way of saying "think like a scientist." Scientists don't assume anything. They start with questions like, What are they absolutely sure are true? What has been proven?" - James Clear
Why Shall We Train our Minds to Think in First Principle?
First Principle thinking helps you dig deeper and deeper until you are left with only the foundational truths and context. It helps to strip out all the common assumptions you may have and come up with something original.
"Usually, when faced with complex problems, we default to thinking like everybody else. First principles thinking is a powerful way to help you break out of this herd mentality, think outside the box, and innovate brand-new solutions to familiar problems. By identifying your current assumptions, breaking these into their basic truths, and creating solutions from scratch, you can uncover these ingenious solutions to complex problems and make unique contributions in any field."
It helps you dive deep, be right a lot, and invent and simply.
In this article, I will share three-step processes you can do to adopt the First Principle framework when you solve complex, abstract problems.
Identify all assumptions
Assumptions are what you think you "know" about a given problem or scenario.
Here are examples of assumptions:
"It requires money to acquire new users on the internet."
"If the payment processing is down, the system cannot proceed further."
"It requires a lot of work to consolidate APIs across multiple departments in the company."
"It is very hard to incorporate 3DS with multiple third-party payment providers because they provide a very different form of API to the merchant."
Assumptions are boundaries and perceptions of the world. It is the "best practice" that most people approve. Though, it doesn't necessarily hold.
Knowing the assumptions helps you understand bias. Understanding the assumptions helps you deconstruct the problem into smaller ones to reach the core problem.
"Business asks about integrating 3DS with various payment gateways. The challenge is that each payment gateway has different ways of integrating 3DS with merchants. Therefore, I assume it is hard to unify them into a single API because they have different API use cases."
You may be surprised that a lot of the time, assumptions are what limit us from solving our problems most efficiently.
Deconstruct the problem into its fundamental
Fundamentals are the fact that you know true by the law of nature and science.
Here is an example of a fundamental:
"Users will want to come back and purchase your product/service. Users will come and use your product if they know about your product. If they don't know about your product, they won't be able to purchase the product. Thus the knowledge of making users come back and use your product is a great experience. The fundamental knowledge of making more users use your product is discoverability."
Different solutions present themselves in different layers of abstraction. Breaking down the problem into its fundamentals strips out all the abstraction layers of the solution. It helps you understand the root cause and all the available options for solving the solution.
Fundamentals help you understand the form of the problems and iterate on that previous form instead of focusing on the current functionality.
You can slowly peel the problem into core problems by asking powerful questions. The one method that I often use is to keep asking myself about the why's and hows to go 2-3 layers deep.
"We need to integrate 3DS with multiple payment providers. Why?"
"If one provider is unavailable, we automatically fall back to another payment provider. Why do we need a fallback?"
"Because fallback can help the system to proceed with the users’ transactions. Why do we want the system to proceed and transact with the users?"
"Because if we are not able to continue. Why do we want the system to continue if such a problem exists?"
"Because we can create a good user experience for someone purchasing our product."
When I was assigned to lead the project of creating a multi-tenant 3DS system for the payment system in Disney Streaming, I encountered a challenge - each payment gateway has different ways of integrating 3DS with merchants. For instance, in Adyen's payment gateway, you will have a three-way flow requiring front-end interaction. In World Pay's payment gateway, you will have a 2-way flow that doesn't require front-end interaction. However, breaking the problem into its fundamentals and understanding that 3DS is a protocol. Because 3DS is a protocol, all third-party payment gateway should implement similar protocol operations. Thus, I looked deeper into how Adyen and Worldpay initiated 3DS as a protocol. From there, I fully understood the fundamentals of the problems and can iterate on that fundamentals to create a new solution.
Start thinking, "What am I sure is true?" "What has been proven?"
Create a new solution based on the fundamental problems
Understanding the fundamentals helps you to alter the direction of the improvements. For instance, back to the 3DS example, since I know how third-party payment gateways implement their protocols and create that protocol abstraction for their merchants, I can understand which part of the process was abstracted on Adyen and which part of the process is abstracted by Worldpay. I realized that the 3DS protocol always does a 3-way process: the authentication flow, the challenge flow, and the authorization flow. I started asking questions to their solution engineers if any other API specs allow us only to do the authentication or authorization flow.
If there is none, can we discuss with the product team to see if they can negotiate with the third-party provider to provide such an API for us?
The core problems help you create new insightful solutions from scratch. It helps you to think outside of the box.
A tip I have when designing a solution deep down to the core problems is to create a list of possible solutions based on the problems at each layer and identify their pros and cons. After writing down the advantages and disadvantages, I look back on the problems and the requirements to see which approach best solves the core problem.
"It is a challenge to integrate 3DS through multiple payment providers because provider A abstracted out the authentication part of 3DS, and provider B abstracted out the authorization part of 3DS. The good news is that 3DS is a protocol that all third-party payment providers must implement. The first approach is stickiness with the user and third-party provider. If user A’s card number is first authenticated through Adyen, they also have to authorize their card through Adyen. The advantage of doing such is that it will be relatively easy to implement. However, the disadvantage is that if Adyen's service is unavailable, the system cannot proceed further. Does that match the core problems that we are trying to solve?"
Conclusion
Solving complex problems requires a long cycle of iterations and improvements. First Principle thinking doesn't remove the need for continuous improvements, but it does alter the direction of the improvements.
Don't start thinking about the solutions when you have presented a complex problem. Try to:
Identify all general assumptions that you have about the problems
Identify the core problems by breaking them down into their basic form
Create a new solution from scratch based on the current constraint
When you solve problems with this thinking framework, you start solving problems by optimizing the form rather than the function (link). Most importantly, you will create more groundbreaking solutions that everyone might not see and create a more positive impact on your team.