In this post I want to describe one form of Systems Thinking that I apply in my work and honestly in many aspects of my life. This is what I call Question and Answer thinking.

Whenever I start investigating an issue, solving a problem, or thinking about a new system to build I start by asking myself a question. I keep a notes document and I write this question down. Then, I take that question as a heading and start trying to answer it.

The first question is usually very broad, extremely broad. If I’m looking a bug in an OpenGL driver implementation in which the wrong image is rendered I simply ask: why did it draw this? If I’m looking at a high latency issue in a component of an AV system I might ask how does this component spend its time?

Are these questions helpful? These questions are obvious: of course we don’t know why the wrong image is drawn or how the time is spent. Or maybe we do know why in which case the question is even less helpful, right?

I find these questions and writing down the questions to be helpful in setting the context of the investigation, or design, or whatever work I’m trying to do. Writing down the questions sets an implicit goal of trying to write down an answer. When we arrive at an answer, writing it down is very helpful in that it serves to

  • Document something true, and apparently not obvious, about our system.
  • Helps to maintain context of what we know about the system
  • Directs further investigation

Ok, so we have a broad, obvious question written down which sets a goal to find an answer and document truth. Great, now what? Well, I usually start by asking a smaller question? Let’s continue with the example of looking at performance in an AV system.

  • Q: How does this component spend its time?
    • Q: What tools do I have for investigating how a component spends its time?
      • A: I have a tracing tool to extract and visualize timing data.
      • I’ve run the tracing tool and generated mycomponent.trace.html.
    • Q: What does the trace say?
      • A: The trace shows a lot of time spent in function foo?

So, this is an example refining questions and getting answers. I have questions and answers, and also maybe some other notes I took along the way explaining what I did or things that were unexpected. Now, I like to do this in a system that allows for folding. So I can fold closed questions. I’ve done this in org-mode and in Google docs. I also like to do a little curating of the answered questions by reordering the question and the answer. That is, when I have an answer I make the answer the headline and put the question under. If I do that I might end up with:

  • A: The component spends a lot of time in function foo.
    • Q: How does this component spend its time?
    • A: I have a tracing tool to extract and visualize timing data.
      • Q: What tools do I have for investigating how a component spends its time?
      • I’ve run the tracing tool and generated mycomponent.trace.html.
    • A: The trace shows a lot of time spent in function foo?
      • Q: What does the trace say?

Now, depending on the tools you are using this curating can be a little tedious. I’ll admit that I found it tedious just now while writing the example. That said, in a real investigation you might spend quite a bit of time, say, reading and interpreting the trace so maybe this tedium is broken up by real work and overhead is low.

The advantage of curating the answers to the top is that when folded it reads like this:

  • A: The component spends a lot of time in function foo.
    • A: I have a tracing tool to extract and visualize timing data.
    • A: The trace shows a lot of time spent in function foo?

On one level, this is as I said before in that the answers document true and not so obvious facts about our system. But when we use this system of refinement, hide the questions, and just look at the answers we have not just a set of true statements but a true statement that answered our big broad question and a series of justifications of that fact. Further, we can open up any level to find notes on how we arrived at these conclusions.

This is very powerful.

Imaging you were working on a hard problem that’s going to take a while and you get pulled off of it to work on some high priority bug. When you come back to the hard problem you can review a series of true statements about the problem that summarize what you know so far organized as a line of reasoning that you believe is directed toward the solution to your problem.

One thing to note, just because I’ve answered the initial broad obvious question doesn’t mean I’m done. In our performance example we concluded “The component spends a lot of time in function foo.” but so what? This doesn’t fix the problem or even really tell us how to fix the problem. But, it might help us figure out how to fix the problem if we ask more questions given our new knowledge. On one hand maybe we should have asked an even broader question first, on the other it doesn’t really matter. I have not objection to adding more broad questions and the top level. I can continue with any or all of:

  • How much time should we spend in the function foo?
  • Are we hitting a particularly bad case in foo?
  • How does the component spend its time in a well performing case?

Answering these new questions will give me a better understanding of the system and what went wrong in this particular issue.

Now, I’ve been doing this for… decades, I guess. And at this point I not only take notes this way but I think this way. This is how I approach problems: I ask a question and either answer it or break the question down into smaller questions until I know the answer or know how to find the answer. Finding the answer can mean

  • Read some code
  • Stop something in a debugger and inspect the state
  • Ask someone
  • Etc.

But if I know how to find the answer I do that. I write down what I found in the debugger or what my coworker told me in the notes and the set the answer. I continue this way until the problem is solved.

Now, this idea of “I ask a question and either answer it or break the question down into smaller questions until I know the answer” is very similar to the way I write software. We breakdown functionality using successive refinement making smaller and smaller functions until the function are easy to implement.

Similarly to writing this code this work strategy is good for large problems. It’s overkill for things that are very simple. Starting a doc, writing down questions, curating answer doesn’t make sense if you already know what is wrong. If you just wrote some new code and now it crashed and you can realize, “oops I passed the wrong argument here and it’s going to difference a NULL pointer in this case” then just fix it. Don’t write these notes and don’t refine any questions. But, if you have a large problem to work on that might take a few days, or longer, then this system can be very powerful and this approach to systems thinking can really help you dig into problems, navigate systems, and build amazing things.