How to make good things

I like to make things that are good. Sometimes I even succeed.

I realised recently that there’s a pattern a lot of my successful attempts (and many of my unsuccessful attempts too. There is no royal road to quality) match that feels different from how other people often seem to go about doing this, so I thought it might be worth writing down and codifying.

As with all such patterns, this post is probably not literally true, but I think it might be quite helpful anyway.

So suppose you want to make something good? What do you do?

“Make” can be a fairly general term here. I use something like this technique for making recommendations to people as much as I do for creating new things. The creative examples are probably more interesting because they’re harder work and easier to get wrong, but I’ll use both for examples.

The system that I have more or less boils down to two main principles:

  • Start with examples of things that are good that you don’t want to make
  • Define rules for your project and stick to them until you can’t feasibly do so

If you want the short, pithy, advice and don’t care about why, you can stop here. The rest of this article is going to be about explaining where these come from, how they help, and how to effectively work with them.

These principles are mostly designed around the desire to take an unstructured creative process and lightly structure it in a way that doesn’t add much cost but tends to improve the outcome.

When trying to make something, I think most people rely on a fairly unstructured process for figuring it out, which results in a plan of attack that looks something like the following:

  1. Decide what “good” looks like for this problem.
  2. Make something good.

It’s not an unreasonable plan of attack, and in many cases it will succeed.

Examples of how applying this strategy looks:

  1. When recommending a book, just suggest a book you really liked.
  2. When writing software, ask people what their problem is and write software that solves that problem.
  3. When writing a story, come up with an idea for it and write that story really well.

All of these more or less succeed or fail on the strength of how well your skills fit the problem and how well you apply them to it: If your taste in books is a good proxy for the person you’re recommending a book to, you will recommend books well. If you’re good at writing software and understanding peoples’ problems you will write software that solves their problems. If you’re a good writer, you will produce a good book.

There’s a lot more to say about it than that, but it’s more or less besides the point of this essay. For now I will take it as read that if you’re good at what you do and you execute it well you will mostly produce something good.

Mostly.

The major problem with this plan of attack for me is novelty.

Often novelty makes the difference between something that is good and something that is merely OK.

Example: I actually really enjoy Elementary as a TV show, but I wouldn’t exactly describe it as good.

Because let’s be honest. We’ve seen this show before.

There’s nothing wrong with making the same thing over and over again. It’s a useful skill building exercise, particularly when you’re starting out, and often as a consumer of stuff I don’t necessarily want new and challenging, you just want something fun that you know you’re going to like.

But even within that some novelty is important:

For learning: Making the same large sort of thing over and over again quickly plateaus as a learning technique. Practising a skill repeatedly only works well if the problem is extremely small so you get fast feedback. At a larger scale if you’re not doing something substantially different you’re learning progressively less and less with each repeat.

For consumption: Even within the brain candy category, I think we overestimate how much we want the same thing over and over again. If we really wanted that we’d just watch reruns or reread the book. You know how an author’s second book is often a lot worse than their first book? It’s probably mostly not. It’s just that the spice that novelty added has worn off and you’re seeing a more “objective” measure of how good their work actually is.

So those are the reasons why novelty is Actually Important. It’s a stronger priority than that for me – I just don’t like retreading well explored ground – but even if you don’t go as far as me on that front novelty is still important.

(side note: This will probably prompt someone to send me a link to prior art for the subject of this post as a hilarious joke. Please do. I don’t read enough on this and I’d like to read more, but it’s hard to separate the wheat from the chaff so I often don’t bother)

So, the key to producing something good is novelty? Great. Let’s make something novel.

Unfortunately just setting out to produce something novel also doesn’t work. If you just seek novelty then you will probably fail at the goal of being good. Novelty is easy: Pick a bunch of random things (use dice or cards or computers because humans are bad at random), throw them together, see what happens.

What will usually happen is it won’t work. I can pick a random book and the person I’ve recommended probably won’t have read it, but they probably also won’t like it. I can implement this Facebook for Dogs in a custom Forth interpreter implemented in C using continuation passing style, but it will probably segfault and even if it doesn’t it’s still Facebook for Dogs. I can write a story about ten random characters saving the world from an attack force of flying cheese graters, but at best you’re going to get an absurdist comedy.

Even at milder levels, novelty can often be a trap: You get recommendations that are more weird than good, half-baked prototypes, or stories where the core idea doesn’t really work. (I have done all these things).

So you need to find a way to produce things that are both novel and good. Neither suffices on their own.

You can’t optimise for two things simultaneously, so you need to figure out a way to combine them. One way would be to allow the two to trade off against each other – some amount of novelty can be sacrificed for some amount of goodness – but I mostly find that produces results where you’re not very satisfied with it on either front.

Instead the way I like to think about this is that novelty should never be the goal, but it can reasonably be a constraint. You are not trying to produce something maximally novel, you are trying to produce something maximally good which is also novel enough.

This matches what we’re trying to do much better: Goodness is the priority, but novelty is a necessity.

And the way to work for this is not to seek novelty, but to have a process that produces novelty automatically while you work on making it good. Fortunately, I have one of those to share with you.

The basic starting point is to take this idea of novelty as a constraint and modify the starting procedure as follows:

  1. Decide what “good” looks like here.
  2. Make something good that is also novel.

Note: This is a bad plan.

The problem with this plan is that the constraint “it should be novel” is too fuzzy. Fuzzy constraints are the enemy – they either get compromised on as you work or you constantly second guess yourself about them. If you try to follow the above plan you will usually end up with something where you’re not really sure it’s good and you’re not really sure it’s novel (regardless of how good or novel it actually is if you’re anything like me on this front). You end up with something mostly like the attempt to optimise for the combination of both but with more self-doubt.

The following refined plan is better:

  1. Decide what “good” looks like here.
  2. Decide how what you make will be novel.
  3. Make something good that is also novel in the prescribed way.

This is better. It solves the fuzzy constraints problem by making them non-fuzzy.

But it’s still not great. The problem is that it forces you to perform most of your novelty up front when you least understand the problem. That’s not really how creativity works – often the most interesting ideas will only occur to you after you’ve been bashing your head against the problem for a while.

But you can get around that quite easily: The idea is to not come up with a specific novel feature out of thin air, but instead to create constraints that when satisfied will automatically produce novelty.

That is, instead of deciding how what you make is going to be novel, you invert it. You decide how it won’t be like the prior art.

You do this by producing a set of principles with which the work must comply, generated by something that looks roughly like the following algorithm:

  1. Pick some prior art you like.
  2. Find something about it you don’t want to emulate. If there’s something about it you actively don’t like then that’s great, use that. If not just find some other way it would be interesting to be different from it.
  3. Create a simple rule that splits the space of possibilities on an interesting axis and avoids that thing.

Iterate this, each time picking prior art that satisfies the rules you have so far, until you’re bored of doing this or you can’t think of anything you like that doesn’t satisfy those rules.

Here are some examples of me doing this:

  1. Recommending fantasy that is not like Tolkien and doesn’t contain elves.
  2. It is really annoying when QuickCheck and similar fail in erratic ways, so Hypothesis takes as a core design constraint that when you rerun a failing test it should immediately fail with the same error as before.
  3. In Programmer at Large, in order to avoid it being like just about every other nerd power fantasy book (which, to be clear, is a genre that I totally read and enjoy as a nerd with power fantasies), the protagonist is specifically designed to not be especially brilliant or competent and just be run of the mill good at their job.

An important thing to note is that you are not picking prior art that is bad. You are picking prior art that is good. It’s easy to pick on examples you hate, but it won’t produce nearly as interesting results. Just trying to avoid being bad is a recipe for mediocrity. The key here is that we’re trying to be good but different.

This solution almost works, but it has two failure modes that it needs adapting to avoid:

  1. You might fail to produce something that is both good and satisfies the rules (either because you couldn’t satisfy all the rules at all, or because you could but the result wasn’t very good).
  2. You produced something good that satisfied the constraints but it turns out to not be all that novel and in fact is quite similar to something that already exists in ways that annoy you.

The second is only really a failure mode in the sense that you’ve produced something no better than the original method: You’ve still produced something good, even if it isn’t novel enough. You might be happy with that. One worry is that you could have produced something significantly less good than you otherwise have without the constraints, but I don’t usually find that’s a problem – often solving in a constrained problem domain is just a good way to make you think about the problem harder and produces results that are better even when they’re not novel. The relation between creativity and constraints is so well trodden at this point that it’s practically a cliche.

This also tends to mean that in the first failure mode the case to worry about is not really “I produced something that satisfied these constraints but it wasn’t very good”. It’s not that that doesn’t happen, but it tends to happen much less often than you might expect. The real thing to worry about is that you’ve made life too hard for yourself to come up with any solution to all the rules – either because none exists or because it’s too hard to find.

With both of these failure modes you can solve this problem by restarting the process and changing the rules – if you couldn’t satisfy them, figure out some relaxed or different rules. If satisfying them produced something very novel you now have a couple new examples to try to exclude!

Depending on how hard the process is, this might be a perfectly reasonable thing to do. If you’re recommending a book or working on a problem you can solve in an afternoon, this is probably fine. With larger things you can also use whatever is left over from the first iteration as as raw material to stop you from having to start completely from scratch.

But even when restarting isn’t that expensive, it’s probably still best not to do it two much.

The “not actually novel” trap is hard to avoid when you don’t have a good idea of what exists. The best fix for this is familiarity with the prior art, but you can also outsource this – if you can’t think of examples satisfying your rules, ask other people for some! Then become familiar with those examples.

For avoiding the trap of creating too restrictive rules, I find that it’s useful to maintain an existence proof as you build your rules: An example that satisfies all the rules you’ve got so far. This can be an existing thing, or a sketch concept of how you could satisfy the rules. When you add a rule you either pick one that the existence proof satisfies or one where you can find a new example satisfying it and all the existing ones.

The reason why this works and isn’t the same as just solving the problem as you go is this: Your existence proof doesn’t have to be good. In many ways it’s often better if it’s bad because that gives you a starting point for the next part of the creative process: Why is this bad and can I fix that?

Instead you can just solve it through brute force: Solve the problem directly in front of you and do the simplest thing that can possibly work without thinking about the bigger picture. All you’re trying to do is show that a solution is possible, not find a good one right now.

I think of a lot of the early versions of Hypothesis as being like this. The modern implementation of Hypothesis only became possible because it passed through a number of intermediate solutions that were kinda awful but demonstrated that a solution was possible.

So, to put this all together:

  1. Maintain lists of rules along with accompanying reference examples, which starts empty, and an existence proof, which is any example of the thing you’re trying to create that satisfies all the rules (regardless of whether it is good or not).
  2. Come up with an example of something you like that satisfies all the existing rules. This might be your existence proof or it might be something else. If you can’t think of one, ask people. If they can’t think of one either, proceed to step 4.
  3. Come up with a simple rule that excludes the example you came up with, along with an existence proof that it can be satisfied along with all your existing rules (this may be your existing existence proof). Add the rule to your list along with this as its reference example, update your existence proof if necessary, then go back to step 2.
  4. Try to create something good that satisfies all the rules.
  5. If you created something good, check if you still think it’s novel. If it’s not, go back to step 2 with it as the example. If its, stop. You’re done. Congratulations, you made a good thing!
  6. If you failed to create something good, try to figure out what the rule or rules at fault were and see if you can modify them in such a way that they still exclude their reference examples but avoid the blockage you encountered. Then go back to step 2.

I very rarely (never, I think, though now that I’ve written it down I might try it) actually sit down and followed the above steps. I’ve probably never even done something that perfectly executed those steps implicitly – I’m much worse at searching the existing literature than that would imply, even though the results are usually better for it when I do – but I think they capture a core process pretty well nevertheless.

And that process is extremely beneficial. It forces you onto a path which seeks out novelty, and by doing so through creating constraints it does a very good job of inspiring the levels of creativity that are required to create that novelty.

It’s also another technique that is very good for learning while doing. The focus on rules and constraints forces you to learn a lot about how the problem space fits together. Although it elides a lot of details in the steps (how do you produce something good within the constraints? How do you come up with a rule?), by giving you more focused questions to answer and explicitly exploring the shape of the problem I suspect you’ll learn a lot better than just seeking to produce something good will.

Ultimately regardless of how closely you follow the specific steps, I think the ideas here are important, and thinking in terms of them will help many and possibly most creative processes.

(And if you found this useful, and would like me to keep making good things on this blog, you’d be very welcome to donate to my Patreon for supporting my blogging to say so regardless of whether that’s novel)

This entry was posted in Open sourcing my brain on by .

One thought on “How to make good things

  1. pozorvlak

    It’s interesting (for me, at least) to read this in light of my recent mountaineering expedition to Kyrgyzstan (I promise I’ll stop going on about it eventually). Our goals were explicitly “novelty” (in the sense of “the mountains we climb should not have previous recorded ascents”) and “within our capabilities” (failure to reach the summit was acceptable, though nonoptimal; failure to get down alive was Right Out). We did a fair amount of reading about prior art (in this case meaning “reports of previous expeditions”), and I guess that fits your approach too, in that it allowed us to identify things we didn’t want for our expedition. In particular, we wanted to avoid the common “problems” of

    – too much bureaucracy (hence Kyrgyzstan, which is much less paperwork-happy than most countries in the region)
    – too much reliance on just showing up and expecting things to work out
    – not enough novelty (hence going to a valley with no recorded ascents, rather than attempting a new line on an existing mountain)
    – too much complexity (guides, porters, mountains of gear – in retrospect, we could have gone more lightweight, but knowing what to leave out is the trick)

    We weren’t explicitly aiming for quality, but I think we managed to achieve it: the actual climbing was by no means difficult or impressive, but it was extremely enjoyable, and the trip as a whole was a great experience. On the other hand, there were several extremely aesthetic lines in the valley that we didn’t attempt because they looked like they might be too hard and/or dangerous: we were very conservative on that front.

    It’s hard to iterate on something which has both long timescales and a definite moment of commitment (“The horses won’t go any higher, so I suggest you camp here; see you in fifteen days!”), but I’d definitely like to do something similar again, building on what we learned and hopefully upping the difficulty a bit.

Comments are closed.