Category Archives: programming

It’s OK for your open source library to be a bit shitty

(Content note: I normally try to keep my natural level of profanity slightly under control on this blog. I won’t be doing that in this post)

The major reason I wrote Hypothesis is to destroy shitty software. Everything is terrible, and I want it to be less terrible.

But that random code you threw together as a hack, stopped when it did what you needed to do, threw it up on pypi and then neglected it?

That’s totally OK. Thanks for writing it. The world is slightly better for your having done so, and there is no burden of expectation on you to “do a better job”. It’s not a job after all, is it? We’re not paying you to do it.

And this is what it ultimately comes down to.

I flatter myself that one of the things that I can legitimately claim about Hypothesis is that it is high quality. So far the worst bug that anyone has reported in the 1.0 release is that when given a wrong argument, Hypothesis throws an exception with the right type from the right place with the wrong error message. Hypothesis works on OSX, Linux, Windows, probably *BSD (I heard about some packaging issues with sqlite, but nothing since, so I think it worked once those were resolved), Python 2.7, 3.2-3.4, CPython, Pypy… It’s got 100% branch coverage, is documented, etc. Basically as far as quality goes Hypothesis does almost everything right.

So I’ve proved it can be done and that you should do it too, right?

Nah, that’s bullshit. I’m here to tell you as someone who has done the work of producing quality software for free, you don’t have to and you shouldn’t feel bad about not doing so.

Want to know how I did everything right in Hypothesis? I mean, obsessive attention to detail and high standards helped, and may even have been essential, but they weren’t even close to sufficient. Really there are two things that were the key ingredients to my making Hypothesis the quality piece of software it is today:

Time
Money

I’ve put somewhere in the region of 800 hours of work into Hypothesis this year, entirely for free. That’s what it took to get to this level of quality.

And I could only do this because I had the time and money to do so. I had the time to do so because I was being obsessive, had no dependents, and didn’t have a job. I could only not have a job because of the money. I only had the money because I spent the latter half of last year with double the salary I was used to, half the living expenses I was used to, and too borderline depressed to spend it on anything interesting.

These are not reasonable requirements.

Could I have done Hypothesis in less than 800 hours? Probably. I doubt I could have done it in less than 400 though, and I would be foolish to expect I could do any project in the smallest amount of time I could feasibly do it in.

Hypothesis is a large and complicated project though (if it doesn’t look like one, that’s because a lot of those 800 hours were spent on making it easy to use). Most projects are probably an order of magnitude simpler.

i.e. only 80 hours.

i.e. only you having to take two weeks off work, working literally for free, in order to produce quality software.

i.e. nearly half your holiday allowance if you live in a civilized country, or possibly more than your holiday allowance if you live in the US.

This is still not a reasonable requirement.

Can you produce quality software in less time than that, working only in your free time? I doubt it. Free time is inherently less productive. You’re tired and it’s fragmented. You spend an hour one evening trying to figure out why your windows builds are now failing because a new version of pip is released and after that I guess you could put another half hour in but your heart isn’t really in it and you’d basically not have the time to get properly stuck into it. The bar for quality is high, and the obstacles to it are higher, and there’s really nothing you can do to fix that other than to put in the time.

Don’t get me wrong. If you can put in the time I will be incredibly grateful to you. I just don’t think you should feel bad if you don’t.

There is no obligation to free labour. Every hour you put in working on your project for free is a gift to the world. If the world comes back to you and says “You are a bad person for not supporting this thing I need you to support” then fuck them. If they want that they should pay you for it, or do it themselves.

(Edit to note: This isn’t of course to say that you shouldn’t ask for features on open source projects. Only that you are not entitled to them. If you politely ask, that’s fine. If the author then says “Sorry, no, I don’t have the time / am not interested / literally any other reason at all”, that’s fine too)

Note: If you liked this piece, there is a follow up you may wish to read.

Fallacies programmers believe about credit cards

Actually I only have one:

The person trying to use this credit card wants to pay in the currency of the country they are currently in

I have encountered this as a problem multiple times recently: I live in Switzerland. I don’t however have a Swiss credit card, and my bank card is not one I can use to pay online. This is a thing I should sort out but in the meantime I would greatly prefer it if when paying on a British site they would give me the option to have my British credit card billed in GBP. Seriously, this is really starting to get on my nerves. You want my money in GBP. My credit card pays out GBP. Let me tell you to bill it in GBP instead of getting me hit with a foreign currency conversion fee by my bank.

Why mathematics makes you better at programming (and so does everything else)

There’s been a bunch of discussion on whether mathematics is programming, whether you need to learn mathematics to program, etc. on Twitter recently. Some of it has been quite heated. So naturally I thought I’d wade right into the hornet’s nest because I’ve got no sense of self-preservation.

There are essentially 4 levels of question going on here:

Is programming basically mathematics?
Do you need mathematics to be a good programmer?
Is there a useful mathematics of programming?
Can learning mathematics make you a better programmer?

The answers to these questions are respectively no, no, yes but you probably don’t care most of the time and yes.

Cool, well, that was easy. Are we done?

Not just yet. I’m going to spend some time unpacking the last of these because it’s the most interesting: Does learning mathematics make you a better programmer?

Well, yes. Of course it does.

So does learning to write. So does learning to cook. So does learning about philosophy of ethics, or about feminism. Learning another human language almost certainly helps you to be a better programmer. Your programming will probably improve if you learn Judo, or knitting, or tap dancing.

I’d go as far as to say that it is unusual to find a pair of skills where learning one will not in some way improve your ability in the other.

Part of this is just that whenever you learn a thing, you’re also learning about learning. You learn what works for you and what doesn’t, and the next time you need to learn something you’ll be that much better at it.

But there’s more to it than that, right? Surely mathematics teaches you more about how to program than just about how to learn things.

Well, yes, but it’s not because by learning mathematics you’re necessarily learning anything directly applicable to programming. Sure, you might be, and you might even be doing the sort of programming to which that maths is directly applicable. But if even if you’re not, learning maths will still make you a better programmer than say, reading Derrida (probably. I haven’t read Derrida so I might be wrong) or learning to cook (Almost certainly. I have learned to cook, and mathematics was way more useful. Your brain might work differently than mine).

Why?

Well, in much the same way that there is a sort of generalised skill of learning that you can improve just by learning things, there are all sorts of other generalised skills. For want of a better word (if you have a better word, do tell) I call them “microskills”. They’re mostly not things that people think of as skills – it’s things like breaking a problem down into parts, abstract thinking, learning to ask the right questions, perseverance when stuck, learning when not to persevere when stuck, and thousands upon thousands of others.

People tend not to think of these as skills because they think of them as innate abilities. Stuff you’re just good at or bad at and you’ve got to cope with that. And I mean, it’s certainly possible that you can be innately good or bad at these (and I may be foolish enough to touch this debate but I’m not touching that one with a barge pole), but regardless of innate ability they all share a characteristic that makes them skills: they get better if you practice them.

Almost every skill you learn will depend on some of these microskills. Which ones is not at all fixed: People compensate by trading off one for the other all the time. You might be less good at working memory, so you practice your organisation skills and keep meticulous notes. You might be easily distractable and lack the ability to methodically push through a problem but make up for it by creative flashes of insight (yes you can learn to have creative flashes of insight, no creativity is not a fixed thing that you can never get any better at).

But it is at least biased. Different skills will encourage you to draw on different microskills. Mathematics will draw heavily on analytical ones – abstract problem solving, hypothesis generation, breaking down a problem into smaller parts, etc. All of these are things that are super useful in programming. So by learning mathematics you’ve developed all these microskills that will later come in super handy while programming.

But, of course, you can develop the same microskills when programming, right?

Wellll….

OK, no. I’m going to go with yes. You will absolutely develop all of the relevant microskills in programming. Learning to be good at programming by doing programming is 100% a viable strategy and lots of people successfully follow it.

The thing is, you won’t necessarily develop them to the same degree. Some you will develop more, some you will develop less. And this isn’t necessarily because the ones you would develop less in programming wouldn’t be useful to be better at.

There is essentially one key thing required for learning to get better at something (HORRIBLE OVERSIMPLIFICATION ALERT): Feedback

Feedback is how clearly and strongly you discover whether you did well or badly at a thing. If you can just try something and immediately find out if it worked or failed, that’s great feedback. If you try something and 3 months later you get a positive result that might be the result of that something (or it might be the result of something else that happened in the intervening 3 months), well… not so much.

Feedback is essentially how you get better because it lets you know that you are getting better, and that’s how you direct your learning. As you get better at a thing, the amount of feedback you tend to get degrades – and this isn’t necessarily because the benefit you’re getting from improving is decreasing (though it often does) it’s often because the benefit is moving further down the line, or into things that are harder to quantify (for example, the benefits of “a sense of good taste” in programming beyond a basic avoidance of complete spaghetti code are often not immediately apparent until you’ve sat atop a mountain of technical debt).

And this is where learning other skills can be super useful, because they provide different feedback loops. Sometimes they even let you develop microskills that appeared completely irrelevant and then once you had them turned out to be super useful (silly example: Strength training to develop core strength. Totally inapplicable to programming, right? Well, yeah, except that it turns out that actually not being distracted by lower back pain while you program is quite handy), but most of the time it’s just that the feedback cycle on them is different and the incentive structure for developing that skill is different – mathematics will lean more on some microskills than programming does and less on others. Similarly it will provide better feedback for some, worse for others.

I could provide some specific examples of which things I think mathematics will help you develop better, but I won’t. It’s very hard to judge these things, and people can argue endlessly about which is better for what. But the specifics don’t matter: Basically all forms of abstract thinking are useful for programming, and the only way that we could reasonably believe that mathematics and programming could provide exactly the same set of feedback loops and incentive structures as each other would be if we believed that they were the same thing.

And that would be silly.

How hard can it be?

There are two types of people in the world:

People who assume jobs they haven’t done and don’t understand are hard
People who assume jobs they haven’t done and don’t understand are easy
People who try to divide the world into overly simple two item lists

Joking aside, there’s definitely a spectrum of attitudes in terms of how you regard jobs you don’t understand.

Developers seem very much to cluster around the “jobs I don’t understand are easy” end (“Not all developers” you say? Sure. Agreed. But it seems to be the dominant attitude, and as such it drives a lot of the discourse). It may be that this isn’t just developers but everyone. It seems especially prevalent amongst developers, but that may just be because I’m a developer so this is where I see it. At any rate, this is about the bits that I have observed directly, not the bits that I haven’t, and about the specific way it manifests amongst developers.

I think this manifests in several interesting ways. Here are two of the main ones:

Contempt for associated jobs

Have you noticed how a lot of devs regard ops as “beneath them”? I mean it just involves scripting a couple of things. How hard is it to write a bash script that rsyncs some files to a server and then restarts Apache?? (Note: If your deployment actually looks like this, sad face).

What seems to happen with devs and ops people is that the devs go “The bits where our jobs overlap are easy. The bits where our jobs do not I don’t understand, therefore they can’t be important”.

The thing about ops is that their job isn’t just writing the software that does deployment and similar. It’s asking questions like “Hey, so, this process that runs arbitrary code passed to it over the network…. could it maybe not do that? Also if it has to do that perhaps we shouldn’t be running it as root” (Lets just pretend this is a hypothetical example that none of us have ever seen in the wild).

The result is that when developers try and do ops, it’s by and large a disaster. Because they think that the bits of ops they don’t understand must be easy, they don’t understand that they are doing ops badly.

The same happens with front-end development. Back-end developers will generally regard front-end as a trivial task that less intelligent people have to do. “Just make it look pretty while I do the real work”. The result is much the same as ops: It’s very obvious when a site was put together by a back-end developer.

I think to some degree the same happens with front-end developers and designers, but I don’t have much experience of that part of the pipeline so I won’t say anything further in that regard.

(Note: I am not able to do the job of an ops person or the job of a front-end person either. The difference is not that I know that their job is hard therefore I can do it. The difference is that I know that their job is hard so I don’t con myself into thinking that I can do it as well as they can. The solution is to ask for help, or at least if you don’t don’t pretend that you’ve done a good job).

Buzzword jobs

There seems to be a growing category of jobs that are basically defined by developers going “Job X: How hard can it be?” and creating a whole category out of doing that job like a developer. Sometimes this genuinely does achieve interesting things: Cross-fertilisation between domains is a genuinely useful thing that should happen more often.

But often when this happens it’s at the expense of the actual job the developers are trying to replace being done badly, and a lot of the things that were important about the job are lost.

Examples:

“Dev-ops engineer” – Ops: how hard can it be? (Note: There’s a lot of legit stuff that also gets described as dev-ops. That tends to be more under the heading of cross-fertilisation than devs doing ops. But a lot of the time dev-ops ends up as devs doing ops badly)
“Data scientist” – Statistics: How hard can it be?
“Growth hacker” – Marketing: How hard can it be? (actually I’m not sure this one is devs’ fault, but it seems to fit into the same sort of problem)

People are literally creating entire job categories out of the assumption that the people who already do those jobs don’t really know what they’re doing and aren’t worth learning from. This isn’t going to end well.

Conclusion

The main thing I want people to take from this is “This is a dick move. Don’t do it”. Although I’m sure there are plenty of jobs that are not actually all that hard, most jobs are done by people because they are hard enough that they need someone dedicated to doing them. Respect that.

If you really think that another profession could benefit from a developer insight because they’re doing things inefficiently and wouldn’t this be so much better with software then talk to them. Put in the effort to find out what their job involves. Talk to them about the problems they face. Offer them solutions to their actual problems and learn what’s important. It’s harder than just assuming you know better than them, but it has the advantage of being both the right thing to do and way less likely to result in a complete disaster.

What is good code?

A long long time ago, in an office building a couple of miles away, I was asked this question in an interview. I’m not sure how much they cared about the answer, but mine was more glib than it was useful (I got the job anyway). I misquoted Einstein and said that good code was code that was as simple as possible but no simpler.

This was not a very satisfactory answer to me, though I got the job.

For, uh, reasons this question has been on my mind recently and I think I’ve come up with an answer that satisfies me:

Good code is code that I am unlikely to need to modify but easily could if I wanted to.

Of course, good code is probably only roughly correlated with good software.