Lessons learned: Are you solving the right problem?

I like to think I’m pretty good at problem solving. How much this is really the case is up for debate, but either way I’ve learned a lot of useful little lessons in the course of testing it.

A really important one is this:

The problem you are trying to solve is often harder than the problem that you need to solve.

Problems frequently come with a set of requirements. It has to play games, let me do my email and make coffee. Well… does it really? Actually it’s much simpler to solve if you make your own damn coffee.

This is a slightly spurious example because the making coffee requirement is so obviously tacked on. But the general idea is common: Of the requirements your problem comes with, one or more may actually be non-essential and dropping it can simplify your life substantially.

Let’s take two real examples:

The first is at work, on the SONAR system. The objective is to build profiles for peoples’ knowledge based on their authored content. So what do we do?

  1. For each document, assign it some tags that describe it
  2. For each user, look at the tags of documents they’ve authored and give them a score for that tag based on that

This is an interesting example of an implicit requirement. We’ve broken the problem up into two steps, and each of those steps has thus become a requirement for the overall problem.

One of the first things I realised when I started working on SONAR is that this implicit requirement was making our life a lot harder than it needed to be. Automatic document tagging with an unknown set of tags is hard. There are lots of papers telling you how to do it, and none of them actually do a very good job (the best of them all tag in relation to some background corpus, but that’s not so good when there are e.g. a lot of internal project names to the company which aren’t present in that corpus).

Given that the end result on users is what matters, this realisation frees us from having to achieve a high precision per document and allows us to focus on recall (Translation: we don’t need to make sure all the results we get are good, we just need to make sure that the good results are contained within them). One minor rewrite later, our theme quality had gone up and our implementation complexity down.

My next example is much more controversial. Which is also a relevant point: Not everyone is going to agree on whether a given feature is essential. I still think I’m right on this one, but then I would. I’ve more or less resigned myself to the idea that the battle is lost here, so this is purely illustrative rather than an attempt to convince.

The example is the 2.8 Scala collections library. The signatures of map and flatMap are quite complicated:

  def map[B, That](f: A => B)(implicit bf: BuilderFactory[B, That, Traversable[A]]): That
  def flatMap[B, That](f: A => Traversable[B])(implicit bf: BuilderFactory[B, That, Traversable[A]]): That

In comparison, an experimental collections API design I put together a while ago called alt.collections has substantially simpler signatures (equivalent to the old signatures in scala.collection really)

  def map[S](f : T => S) : Traversable[S]
  def flatMap[S](f : T => Traversable[S]) : Traversable[S]

Why is my version so much simpler than Martin’s? Well, as much as I’d like to claim it’s because I’m an unparalleled genius of API design compared to Martin, that’s really not the case.

The reason for the comparative simplicity is that Martin’s version does something that mine does not.

The problem we’ve both set out to solve is that the return types of map and flatMap are very inconsistent in the current API: Sometimes they’re overridden to return the right type, sometimes they’re not (sometimes they can’t be). This upsets people. Essentially there is the following requirement that isn’t being met by the current collections API:

map and flatMap on a collection should return the same type of collection

Martin responds by meeting that requirement. I respond by taking it away. The design of alt.collections was centered around the idea that map and flatMap would not attempt to return the “right” type but would instead always return a Traversable. It then set about making it easy to get the right type out at the end.

More than one person has responded to this design with “I’m sorry, but that’s horrible” or similar, so clearly a lot of people think in this case the answer is “Yes, we are”. But I still find it a rather compelling example of how much simpler you can make the solution by first simplifying the problem.

This entry was posted in life, programming on by .

5 thoughts on “Lessons learned: Are you solving the right problem?

  1. Jesper Nordenberg

    Hmm, I’m a bit puzzled how Martin’s versions even work. If I’m not mistaken a type parameter can’t be inferred from an implicit parameter, so how is “That” bound to the correct type at call site? Will it require explicit binding or is the type inferrer improved in 2.8.0?

  2. Dan

    I love it, but I have two problems.

    1) You’re quite lucky in that you can push back on the requirements, as in your document tagging example. I’ve never been so lucky: the mere suggestion of simplifying leads the analyst/salesperson/customer/manager to begin to have epileptic fits, followed quickly by frothing at the mouth, fainting spells, and the stammering of thousands of reasons why the solution must literally be the hardest possible one or it will do no good.

    2) Since no rationalization works for simplification of requirements, I attempt simplification by assumption-reduction. I try to treat problems like Sun-Earth-Moon problems, where I can ignore the effects of the moon to get into the ballpark then add it back in to improve my results. But, as I’ve learned the very, very hard way, in places where there “are” thousands of reasons for extreme complexity, every problem is an n-body problem.

    I’m sure you’ve encountered these; how do you deal with them?

    To be clear, I don’t deny your hypothesis that reducing complexity in the problem makes a linear-plus improvement in the solution complexity. It’s my strong agreement with it that leads me to continually bang my head against the walls of (1) and (2).

  3. david Post author

    Right. When the requirements are handed to you by someone who you can’t interact with on a technical level you may unfortunately end up with cases where you might as well try to move mountains as drop requirements. It’s sad. :-(

    The document tagging example has the pleasant feature of being purely technical: From the end user point of view it simply has no visible effect except that their themes suddenly get a lot better. So from their point of view no requirements have been dropped! So this advice is still useful even in the case where the external functionality is written in stone.

    One way that can sometimes help with the client facing interaction is to simplify by separating. e.g. to take my silly coffee example “Well, I *could* build you a thing which does that. But how about I buy you a computer and a coffee maker instead?”. i.e. rather than trying to simultaneously satisfy the full set of requirements you offer to satisfy sets of them separately. But I must admit to having only limited experience with trying this in practice.

    I also tend to solve the problem by evading it. These days I’m very much in the technical guts of the software rather than on the very user facing side. And I love it that way. :-)

  4. Pingback: Best of drmaciver.com | David R. MacIver

Comments are closed.