Category Archives: programming

It might be worth learning an ML-family language

It’s long been a popular opinion that learning Haskell or another ML-family language will make you a better programmer. I think this is probably true, but I think it’s an overly specific claim because learning almost anything will make you a better programmer, and I’ve not been convinced that Haskell is a much better choice than many other things in terms of reshaping your thinking. I’ve never thought that you shouldn’t learn Haskell of course, I’ve just not been convinced that learning Haskell purely for the sake of learning Haskell was the best use of time.

But… I’ve been noticing something recently when teaching Python programmers to use Hypothesis that has made me reconsider somewhat. Not so much a fundamental reshaping of the way you think as a highly useful microskill that people seem to struggle to learn in dynamically typed languages.

That skill is this: Keeping track of what the type of the value in a variable is.

That may not seem like an important skill in a dynamic language, but it really is: Although functions will tend to be more lenient about what type of value they accept (is it a list or a tuple? Who cares!), they will tend to go wrong in interesting and confusing ways when you get it too wrong, and you then waste valuable debugging time trying to figure out what you did wrong. A good development workflow will typically let you find the problem, but it will still take significantly longer than just not making the mistake in the first place.

In particular this seems to come up when the types are related but distinct. Hypothesis has a notion of a “strategy”, which is essentially a data generator, and people routinely seem to get confused as to whether something is a value of a type, a strategy for producing values of that type, or a function that returns a strategy for producing the type.

It might be that I’ve just created a really confusing API, but I don’t think that’s it – people generally seem to really like the API and this is by far the second most common basic usage error people make with it (the first most common is confusing the functions one_of and sampled_from, which do similar but distinct things. I’m still trying to figure out better names for them).

It took me a while to notice this because I just don’t think of it as a difficult thing to keep track of, but it’s definitely a common pattern. It also appears to be almost entirely absent from people who have used Haskell (and presumably other ML-family languages – any statically typed language with type-inference and a bias towards functional programming really) but I don’t know of anyone who has tried to use Hypothesis knowing an ML-family language without also knowing Haskell).

I think the reason for this is that in an ML family language, where the types are static but inferred, you are constantly playing a learning game with the compiler as your teacher: Whenever you get this wrong, the compiler tells you immediately that you’ve done so and localises it to the place where you’ve made the mistake. The error messages aren’t always that easy to understand, but it’s a lot easier to figure out where you’ve made the mistake than when the error message is instead “AttributeError: ‘int’ object has no attribute ‘kittens'” in some function unrelated to where you made the actual error. In the dynamically typed context, there’s a larger separation between the observed problem and the solution, which makes it harder to learn from the experience.

This is probably a game worth playing. If people are making this error when using Hypothesis, they I’d expect them to be making it in many other places too. I don’t expect many of these errors are making it through to production (especially if your code is well tested), but they’re certainly wasting time while developing.

In terms of which ML-family language to choose for this, I’m not sure. I haven’t actually used it myself yet (I don’t really have anything I want to write in the space that it targets), but I suspect Elm is probably the best choice. They’ve done some really good work on making type errors clear and easy to understand, which is likely exactly what you need for this sort of learning exercise.

 

This entry was posted in programming, Python on by .

Contributors do not save time

(This is based off a small tweet storm from yesterday).

There’s this idea that what open source projects need to become sustainable is contributors – either from people working at companies who use the project, or from individuals.

It is entirely wrong.

First off: Contributors are great. I love contributors, and I am extremely grateful to anyone who has ever contributed to Hypothesis or any of my other open source projects. This post is not intended to discourage anyone from contributing.

But contributors are great because they increase capabilities, not because they decrease the effort required. Each contributor brings fresh eyes and experience to the project – they’ve seen something you haven’t, or know something you don’t.

Generally speaking a contribution is work you weren’t going to do. It might be work you were going to do later. If you’re really unlucky it’s work you’re currently in the process of doing. Often it’s work that you never wanted to do.

So regardless of what the nature of the contribution, it creates a sense of obligation to do more work: You have to deal with the contributor in order to support some work you weren’t going to do.

Often these dealings are pleasant. Many contributions are good, and most contributors are good. However it’s very rare that contributions are perfect unless they are also trivial. The vast majority of contributions that I can just say “Thanks!” and click merge on are  things that fix typos. Most of the rest are ones that just fix a single bug. The rest need more than the couple of minutes work (not zero work, mind you) that it took to determine that it was such a contribution.

That work can take a variety of forms: You can click merge anyway and fix it yourself, you can click merge anyway and just deal with the consequences forever (I don’t recommend this one), you can talk the contributor through the process of fixing it themselves, or you can reject the contribution as not really something you want to do.

All of these are work. They’re sometimes a lot of work, and sometimes quite emotionally draining work. Telling someone no is hard. Teaching someone enough of the idiosyncracies of your project to help them contribute is also hard. Code review is hard.

And remember, all of this is at the bare minimum work on something that you weren’t previously going to do just yet, and may be work on something that you were never going to do.

Again, this is not a complaint. I am happy to put in that work, and I am happy to welcome new contributors.

But it is a description of the reality of the situation: Trying to fix the problems of unpaid labour in open source by adding contributors will never work, because it only creates more unpaid labour.

This entry was posted in programming on by .

Fuzzing through multi-objective shrinking

This is an experiment I’ve been running for the last couple of days (on and off and with a bunch of tinkering). It was intended as a prototype for using glassbox in a next-gen version of Hypothesis, but it’s proven interesting in its own right.

The idea is a specific automated way of using a test case reducer as a fuzzer using branch instrumentation (I’m using afl‘s instrumentation via the afl-showmap command): For every branch we ever observe the program taking, we try to construct a minimal example that hits that branch.

This will tend to produce interesting examples because you throw away a lot of extraneous detail that isn’t required to hit that branch. This is is particularly true of “tolerant” parsers which try to recover from a lot of errors.

How it works

The core idea is that we take a normal test case reducer and repeatedly apply it in a way that automatically turns it into a multi-objective reducer.

Say we have a function, label, which takes a binary string and returns a set of labels. Labels can be anything, but in the case of using AFL’s instrumentation they’re essentially branches the program can take along with a rough count of how many times that branch was taken (essentially because the branches are hashes so some different branches may end up equated with each other).

We replace the labelling function with a side-effectful version of it which returns the original results but also updates a table which maps each label to its “best” example we’ve seen so far. We consider a string better than another if it is either shorter or the same length but sorts lexicographically before the other (when viewed as a sequence of unsigned 8 bit integers).

We then repeatedly iterate the following process: Pick a label, take the best example for that label, and reduce that test case with respect to the condition that it has that label (updating the other test cases with every call).

There are various heuristics you could use to pick a label. The ones I’ve tried are:

  • Pick one of the labels which currently has the best example
  • Pick one of the labels which currently has the worst example
  • Pick any label, uniformly at random

Uniformly at random seems to work the best: The others have a tendency to get stuck. In the case of ‘best’ there are a lot of small labels and it ends up spending a lot of time trying to shrink them all, not doing very interesting work in the process. In the case of ‘worst’ it tends to spend all its time trying to shrink very hard to shrink labels and not getting very far. Uniformly at random seems to consistently make progress and find interesting results.

Optimisations

There are a couple of extra useful things you can do to speed up the process.

The first is that every time label is called you can mark the string as known. Then when shrinking instead of shrinking by whether the string has the label, you shrink by whether the string is either the current best for the label or is unknown.

This works because if the string were simpler than the current best and already known, then the current best would already have been updated to that string.

This is the equivalent of caching the predicate for delta-debugging, but you don’t want to cache the label function because its outputs are complex values (they’re sets of labels, so there are \(2^n\) distinct values even after interning) so end up consuming a lot of memory if you cache them.

The second is that you can often tell when a label is going to be useless to shrink and skip it. There are two things you can do here:

  • If when you tried to shrink a label it made no changes, you can mark that label as ‘finished’. If another shrink later improves the label, you remove the finished mark. A finished label cannot be shrunk further and thus can be skipped.
  • By maintaining a counter that is updated every time a label is improved or added to the table, you can tell if an attempt to shrink did anything at all by checking the counter before and after. If it did nothing, you can mark the string as finished. Any labels whose current best string is finished can also be skipped.

This also gives a way of terminating the fuzz when there’s nothing left that’s discoverable: If every label is skippable, you’re done.

Observations

This seems to work quite well in practice. Starting from a relatively large initial example, it quickly increases the number of labels by about an order of magnitude (some of these are just difference in branch counts, as AFL counts not just whether the branch was hit but also a bucketed version of how many times).

It also works pretty well at finding bugs. I’ve been running it for about 48 hours total (a bit longer by clock time but I turned it off in the middle while I made some changes) and it’s found two bugs in a widely deployed file format parser that’s been stable for a couple of years (I’ve sent both to the author of the parser, and don’t want to say which one it is until I’ve got permission to do so. I don’t think either of them are security issues but hard to say for sure). One of them is confirmed novel, and I haven’t heard back about the other one yet. It found the first one after about 10 hours, but that appears to have been mostly luck – rerunning with a bunch of changes that otherwise improved the process hasn’t refound that bug yet.

Anecdotally, almost all of the examples produced are not valid instances of the format (i.e. the tool under test exits with a non-zero status code). This isn’t very surprising: The expectation is that it will give you just enough of the file to get you to the point you’re looking for and then throw away the rest, which is unlikely to get you a valid file unless the branch you’re looking for is taken after the file validity has already been determined.

Comparison with AFL

In some ways this is obviously quite similar to AFL, given that it uses the same instrumentation, but in other ways it’s quite different. My suspicion is that overall this approach will work better as an approach to providing a corpus to AFL than it will just on its own, but it’s surprisingly competitive even without that.

In particular it seems like it hits an order of magnitude increase in the number of seen labels much faster than I would expect AFL to. I think it helps that it’s using AFL’s instrumentation much more extensively than AFL itself actually does – AFL just uses the instrumentation for novelty detection, whileas this approach actually treats each label as a target in its own right and thus can take much more advantage of it.

The AFL algorithm is roughly just to repeatedly iterates the following:

  1. Pick an example from the corpus and mutate it
  2. If the mutated example exhibits any labels that we’ve not previously seen, add it to the corpus

It’s not really interested in the labels beyond novelty detection, and it doesn’t ever prune the corpus down to smaller examples like this does.

This approach also has a “self-optimizing” character that AFL lacks: Because AFL never replaces examples in its corpus, if you start with large examples you’re stuck with large examples forever. Because of this, AFL encourages you to start with very small, fast examples. This approach on the other hand will take whatever large examples you throw at it and will generally turn them into small examples.

To be clear: This isn’t a better approach than AFL. Maybe if it were highly optimized, tuned and refined it would become at least as good, but even then they would both have strengths and weaknesses compared to each other. But it’s not obviously a worse approach either, and even now it has some interesting advantages over the approach that AFL takes.

This entry was posted in programming on by .

Writing libraries is terrible

My talk, Writing Libraries is Terrible, is now available online.

You can see the video on youtube, and the slides here.

Transcription

So, yeah. I’m here to tell you about writing libraries and why it’s terrible. And I’ve been basically working full time on a library for about the last year and a half, so I’ve got to experience all of this first hand.

Some caveats: I’m talking about my specific experience, so I’m talking about writing open source libraries. Closed source, proprietary, libraries, do exist. I have no experience with this at the moment, and my impression is that using those is terrible, so this is what the other end looks like.

I’m also talking specifically about my experience writing a Python library. I think very little of what I have to say is actually about Python, or Python specific, but that does colour the experiences. Different languages and different communities do respond differently.

Now, despite the title, I actually really like writing libraries. You get to focus on a very constrained problem, you get to do a lot of interesting things in the course of doing them. There’s really a lot to recommend writing libraries, and the process of writing them is enlightening and enjoyable.

But then people start using them, and that’s where things start to go a bit wrong.

This was originally going to be a talk full of technical war stories, and I’m really glad it no longer is because Jim completely topped any technical war stories I can give you, but I am going to start by just mentioning the single biggest technical problem that gives me a headache as a library author, which is that I currently support three operating systems officially and really dropping any one of these is completely non-optional [ed. I of course mean “supporting” rather than dropping], and on top of that I have to support a really large number of versions of python, and even with the ones I support people complain about the ones I don’t support. So apparently it’s really annoying that I don’t support Python 2.6 despite the fact that Python 2.6 has been end of lifed for a large number of years.

And in general, everyone’s computer is different, and as an application developer you have to deal with this and it turns out that as a library developer you have to deal with this more, because you don’t just have to worry about what’s on their computer you have to worry about the specific versions of different things. If you’re writing an application you can ship a language runtime with it and not really care which version they already have, there all sorts of advantages. I don’t get any of those.

OK, that’s all I want to say on the technical side of things, because it turns out that none of these are actually the really bad thing about writing libraries. The really bad thing about writing libraries is money.

Because, culturally we’ve just moved to this idea that of course libraries are things we get for free. Like I said, open source libraries are great to use, so of course everyone is using open source libraries. And you do get paid for open source projects, like a lot of the big ones in particularly are very well paid these days. Most people who work on Linux are paid to do so. But a lot of the open source ecosystem isn’t this sort of large open source project, it’s these single serving libraries that do one specific thing and people just install them, they don’t even really think about paying for them.

And what they pay for them instead is complaining. Like I said, people complain at me for not supporting Python 2.6, and I get some really entitled comments about it, basically just like “I’m a giant bank building legacy software in Python 2.6, making lots of money, why aren’t you supporting this thing for my for profit business for free?”, and this isn’t much fun to be on the receiving end of obviously, and it turns out to be quite hard to pay your bills with other peoples’ whining.

But actually none of this is the worst about not getting any money to work on these. Because it turns out that as well as sort of being intrinsically useful, being paid for your work gives you lots of advantages that not being paid for your advantages doesn’t have. One of the biggest, I think, under appreciated advantages of making money is that it gives you an amazingly good decision heuristic. Whenever you come to a choice, and you’ve got a choice where you’ve crossed off all the other directions – you’re ethically OK with both options, both options seem like good ideas, so on and so forth… which one of them is going to make you more money? Do I support this obscure version of Python? Well, am I going to make more money than it costs me in effort to support this version of Python? And in the course of design there are a million choices you can make in all these, and basically with open source, and with this sort of library, all that really guides your choice is do I feel like doing it and will I feel bad about not supporting this. And this turns out, if you’re like me, to be a great way of creating an amazingly large amount of work for yourself, and it’s really not obvious which of the work is important and which of the work is not important.

The other great thing about money is that you can hire people. And you do get contributions as an open source library. My project, Hypothesis, I have absolutely got contributions from people, and it’s great, but this is the amount of Hypothesis I’ve done [ed. I’m holding out my arms wide here] and this amount of Hypothesis literally everyone else has done [ed. My hands are close together now]. And I don’t hold that against them. Why should they? I’m not paying them, and basically everything they do for Hypothesis is a gift, and so I’m very grateful to those gifts, but at the same time, it’s still just me doing most of this work.

And what that means is that I have to be a lot of people. If you think about your normal software projects, you don’t just have developers, you certainly don’t just have one developer. That would be ridiculous, why would you do that? I mean, you’ve got not redundancy there, right? But you’ve also got project managers, you’ve got customer supports, you’ve got sales, you’ve got marketing, you’ve got technical writing, you’ve QA… I have to be all of those people. And… in some ways this is a good thing. I’ve really enjoyed learning a lot of these, and the best way to get a lot of appreciation for your colleagues is to try and do their jobs and discover that all the things that you thought were really easy and you thought your colleagues were kindof idiots and were just making a big deal of it… it turns out that actually people have specialised skills and some of those specialised skills are really hard. I’ve been doing a lot of attempts at marketing recently [ed. This slightly overstates the case. I’m talking mostly about hypothesis.works and some behind the scenes stuff, but I’ve been reading more] and it turns out that maybe those marketers did actually know something. This is not an opinion I would have had a year ago. But at the same time, this is really bad, because I’m not good at marketing and I’m doing the marketing for this project. I’m OK at writing, well I think I’m good at writing but I’m not good at technical writing. Documentation it turns out is an entire new skill on top of both technical understanding and writing. So I’m having to learn all of these because I don’t really have another way to do it, and I certainly can’t afford to pay anyone else to do it.

And this is all a massive problem and this isn’t going to change, because from the point of view of the people using our work, what we are basically doing is giving them money. We’re leaving this giant pile of cash on the ground and they’re just coming along and picking up because someone’s just left this giant pile of cash and obviously that means it’s free for use, right? If someone tells you that there’s this thing that is better than thing you were going to pay for and you can get it for free then of course you’re going to take it.

And this very short term thinking. As we saw in the lightning talk about CentOS yesterday, this will bite you, but by the time it’s bitten you we’ve already done all the work and it’s probably too late. So regardless of what a bad decision this may or may not be on the company’s side, this doesn’t actually help me. And this isn’t just about me obviously, this is about everyone else who also finds themselves in this position. But… in particular this is about me. And so this doesn’t help me.

And there’s really only one way this can change, which is my side and our side. At some point we’re going to have to start withdrawing our labour. If the problem is that people are taking our work and using it for free and this is really bad for us, maybe we could just… stop giving our work away for free? Is that a valid plan? It seems like a valid plan. Every time I try and explain my situation to my family they look at me like I’m crazy and ask me why I’m giving away stuff for free.

And I feel sad about this. I really like sharing my work. I’m not writing open source software because I want to make lots of money. My judgement isn’t quite that bad. I’m writing open source software because I want people to use it, and I want so share these ideas, and I really enjoy doing it.

So what I’m really hoping is that there’s some compromise we can do here. And… software licensing is a large part of this. I don’t know what part the GPL necessarily has to play in any eventual solution, but I do know that the MIT and BSD style solutions have nothing to play in this solution. They can go to hell as far as I’m concerned. And I think there’s more scope for improving the situation with licensing. And I don’t know what yet that’s going to be. I am talking to people to find out what it’s going to be. Because ultimately we do need to give people a better way to pay us, and we do need to still have ways of giving things away for free, and I don’t know where the middle ground is.

But… despite my very adversarial framing of this as us vs them, I don’t think us vs them is the solution either. Because as well as sharing, I also like making money, and I’m perfectly happy for other people to make money, and I think working together is the only way we can move forward. We have to start this, but we can’t finish this alone.

And ultimately as well as being better for us open source developers, I think it’s also going to be better for companies. Because when you think about it, 90% of the code you’re running in production has this label on it, and that’s probably a bad idea.

Thank you very much. Any questions?

[ed. I was stupid and forgot to repeat questions and we didn’t have individual microphones. I’m guessing at what people asked me based on interpretation, memory and my responses]

Question: “You must have had a plan for making money off open source software. What was it?”

Answer: I had various plans. Right now I’m mostly doing services and training. I’m going to be starting offering proprietary extensions which try and make it easy to use the core libraries for everyone, but when you start to have problems that you only really have if you’re a large company who is doing serious deployments of these things then you need to start paying me money. And there are a variety of other plans, I’m happy to talk about that at greater length afterwards. But the short answer is that it’s really hard and no-one seems to have a very successful plan yet. The best plan seems to be to do a really bad job of writing the library and then to get people to pay you to help them figure it out, and I’m not a big fan of that one either.

Question: (I have no idea what was said. I think this was mostly a question about bounty source and similar bounty programs?)

Answer: So for about the last six months a lot of the issues and feature requests on my github tracker have had the tag “for a modest fee” on them. Literally no-one has ever approached me about this. I do have a bounty source account which I need to make more obvious but I currently money from donations on that and zero money on issues. This works very well for some people but there’s this general pattern where applications with lots of consumer usage do quite well on these bounty programs more than things with companies using them. For example I know that neovim seems to be doing very well out of bounytsource right now, and I think it’s again a thing where large open source applications work much better for this than these sorts of small libraries. That is probably part of a viable future but I don’t what part it is and it hasn’t been working for me so far.

Question: Do you feel you should be paying for Python and the standard library.

Answer: Well right now I’m not a successful commercial user of it. I kinda do though, yeah. I think certainly in terms of donating to the Python software foundation that’s something I will probably do once my revenue stream starts to look slightly less pitiful and I actually start paying myself again [ed. I am now paying myself again! Or at least, I am starting doing pay roll for myself and that will at some point actually make it into my bank account], but it’s hard. There’s this huge ecosystem of open source and I don’t know yet where the money for it should come from, but I do feel that a lot more of it should come from the people who are making literally billions of dollars running off it than necessarily the individual developers who are building on top of it. But yes, it’s complex, I don’t know the answer.

Question: (Something about developers not general being buyers in a company)

Answer: Not really. I call this the “take me to your leader” problem, which is that basically the only solution I can figure out is that you need to get developers to introduce you to their managers, but that runs into the developers are not marketers problem because basically you’re trying to turn developers into your marketers and we’re not very good at that.

Question: (I have no idea)

Answer: I think there does need to be better charitable organisations but I think I’m not necessarily the person to come up with the ideas around that or to try that.

Question: Have you approached the Python Software Foundation for funding? (Something more to it than that but I’m not sure what)

Answer: The Python Software Foundation have very specific policies which is that you can apply to them for funding to work on a specific isolated chunk of work, so if there were a feature I wanted to add to Hypothesis that I wanted funding for I could potentially apply to the PSF for it, but for the general problem of ongoing Hypothesis development that’s not the sort of thing that they fund, and so I’ve largely not looked into it past that point. But for certain things that’s absolutely a thing you should think about doing. And also there are within specific areas there are other funding bodies, so for example for Django there’s the Django foundation who can do this, there’s NumFOCUS who tend to focus on scientific Python, and a bunch of relatetd things. So sometimes that works, but it tends to be more solutions to specific problem than the general larger problem.

Question: (I think about the idea of my writing a book)

Answer: Yes I’m going to look into that, because I’ve already a huge number of articles and so I figure I can turn those articles into a book. But that’s more marketing than it is going to be necessarily a useful income stream. I’m expecting that to make some money, and this is the sort of solution that only works once you’ve put in all this work and done all this free work, so again I think writing books is… it’s a great partial solution to this problem but it doesn’t actually solve the problem. And maybe with all the partial solutions available if you do literally all of them it starts to add up to a whole solution, but right now I’m not convinced it does.

Question: Is open source possible the wrong answer to this? Or is just a marketing strategy?

Answer: I don’t know if open source is the wrong answer. I think that closed source is definitely the wrong answer, and I think that finding some hybrid between the two, but as we saw in Jim’s talk earlier and in plenty of others, there’s a reason why open source is so popular and it’s not just that stuff is being given away for free. Your life tends to get very bad if you build your things on a foundation of other peoples’ closed source products. And this sort of free sharing of ideas really does have a lot of benefits, so I don’t want to throw the baby out with the bathwater. I really like open source and want it to succeed, and that’s why I’m trying to find a compromise. In terms of the question of whether it’s marketing or not, marketing is definitely a component but I think we need more than just marketing. I think no matter how well you market your open source project that’s not going to make you any money unless you also have a path to making money from it. Which is what I’m trying to figure out.

Question: Something about using the open source product as the marketing for something.

Answer: That is the open source core, or something like that, and I think that’s a viable solution, but you do then end up with… in that case you’re sort of asking “how much of the benefits of open source do you want to get?” and I would like something where you could get both the benefits of open source and the benefits of commercial work. I would also like a pony.

This entry was posted in programming on by .

In defence of developer exceptionalism

Epistemic status: I am thinking through a possible position out loud. While I believe the argument I am making in this post, I am not totally convinced by it. I am aware of a number of counter-arguments that I do not touch on here.

There’s a rather annoying attitude that often comes up: Software Developers are special and you should treat them differently from everyone else when designing how your organisation works.

I don’t think this is true. Software development is knowledge work with a fairly logical (though less logical than we like to pretend) basis, and most of the specific features of it as a profession just emerge out of that.

But I am starting to think it might be useful.

There is a category of argument I have often encountered in negotiation (not at all coincidentally, usually right before I turned down a job offer or shortly before I left a company). I call it the “We can’t treat you fairly because then we’d have to treat other people fairly too” argument. e.g. I’ve seen this in requests for flexible working (I like four day weeks a lot, and will in general happily take 80% of the salary for a 4 day week) where the response to the request is “But loads of people want that and if you had it they’d ask why they can’t have it too.”

My attitude to this in the past has always been “Well, why can’t they have it too?”

This is, of course, still my attitude, but I’ve come to accept that it will win the argument precisely 0% of the time.

The problem with accepting arguments of the form “Why should you get this treatment if everybody else does not?” is that they remove all possibility of incremental change. Even when everybody should get something, there’s probably no route to that that doesn’t pass through some people getting it and not others.

One example of where this comes up in a more explicitly developers vs non developers scenario is in responses to things like “Why you should not interrupt a programmer“. Every time this or similar is posted, a lot of my friends become angry and say “Programmers aren’t special! You shouldn’t interrupt anyone!”.

This is more or less true (there are large categories of jobs where it’s not only fine but desirable to interrupt them because that’s what their job is. But for basically all knowledge work and a whole bunch of adjacent jobs it’s true). You shouldn’t interrupt anyone.

But it’s a lot easier to win arguments about not interrupting programmers.

And, perhaps, it’s a lot easier to win arguments about not interrupting other people once you’ve established a norm that it’s not OK to interrupt programmers?

And this is why I think developer exceptionalism might be useful. Not because developers are exceptional, but because it is a place where we can put the thin end of the wedge.

A lot of how business is run is very dysfunctional, and exceptionalism gives us a convenient way of side stepping that.  If your business is very interrupt driven but you can establish a small island of uninterrupted calm around your programmers, maybe later the designers can come join you on the island? And maybe after that marketing can note that actually a lot of their work involves pretty intense thinking like that and they’d like to be uninterrupted at least some of the time and…

I can’t promise that it will work, but I’m pretty sure it won’t work without. Systems are quite stable without something to shake them up, and without some specific argument that you should be treated specially, you’ll be treated the same way as everyone else: Badly.

Exceptionalism and solidarity feel fundamentally at odds, and perhaps they are, but perhaps solidarity without the permission to occasionally engage in exceptionalism is just a crab bucket, and the best way to get everyone to where you want them to be is to let some people get ahead by whatever means necessary and then pull the others up behind them?

Of course, if this is going to work you have to actually pull people up behind you.

The correct progression is:

  1. “You can’t have X”
  2. “But developers are special!”
  3. “OK fine developers can have X”
  4. “You can’t have X”
  5. “But the developers have X and it worked really well at improving Y which is also our problem”
  6. (Developers chime in) “We can totally back that up. This is what we’ve seen out of X and it really does help with Y”
  7. “Oh OK, you can have X too”

An incorrect progression would be:

  1. “You can’t have X”
  2. “But developers are special!”
  3. “OK fine developers can have X”
  4. “You can’t have X”
  5. “But the developers have X and it worked really well at improving Y which is also our problem”
  6. (Developers chime in) “No! X is only for developers! You are not a developer and thus are not special and cannot have X”
  7. “You can’t have X”

If developer exceptionalism is to be a useful lie then we need to do two things:

  1. Never forget that it is a useful lie and not actually the truth.
  2. Have other peoples’ back when they try to follow our lead.

Otherwise it’s just developers being privileged jerks, which we have quite enough of already.

 

This entry was posted in life, programming on by .