Author Archives: david

I should edit more

As may be obvious, my blog is only very lightly edited. The normal course of writing a blog post is that I sit down, spew out one or two thousand words that I’ve mostly already vaguely sorted in my head, do a quick once over for anything that is obviously terrible and click publish.

I’m going to experiment with holding back drafts a bit longer and editing them some more, but there’s still this large backlog of pieces that I’ve written that could really use some more love.

So I’m starting a parallel project to this blog: The book of DRMacIver (It will almost certainly never become an actual book).

It’s basically a collection of essays from this blog that I like which I’ll be editing over time to try to raise them to a higher standard of quality. Some of them will be verbatim copies of blog posts, some of them will be essays I’ve extracted (there are a few blog posts which are clearly several smaller blog posts trying to get out), some might be mergers of various posts. We’ll see. New content will continue to be published to this blog first, but updated versions will start to appear over there.

Apologies for the styling. Visual design and CSS are not numbered amongst my skills. I may make an attempt at improving that too, but no promises.

Also, if you want to follow along with the edits, it’s just a publicly available git repository.

This entry was posted in Uncategorized on by .

You should write more

Note: There is an updated version of this piece here. It’s probably not very different, but has had some more editing done to it.

This is an advocacy piece. I would like you to write more because I would selfishly like to read what you have to say – if you don’t have a blog, create one. If you do, update it more often.

Based on conversations with friends I’ve had, I know there are a lot of people with really interesting opinions and viewpoints who never flesh them out into the great long form piece that I know is  in there. I would like them to flesh them out because I want more interesting essays to read.

So, yeah, that’s my purely selfish motivation for writing this piece, but everything I’m saying in it is something I 100% believe, so please ignore my ulterior motives and instead listen to what I have to say: I think you should write more.

Why?

I firmly believe that this blog is the best thing I do.

In two senses really: Firstly, it arguably has the most positive impact on the world of anything I do, but also it’s the most self-improving thing I do.

The first one is debatable and hard to measure. There are a few charities I give money to (not enough. Fixing that is on my should-do-but-probably-won’t list, and will probably remain there until I get my act together) that could arguably count. Work sometimes counts, but it tends to be either a large chunk of a company that isn’t making a major positive difference (or possibly is but in a niche I don’t care that much about) or a tiny fraction of something making a modest positive difference. I’ve written a bit in the past about the positive impact I try to have with this blog, but honestly if this blog disappeared from the earth a few people would be sad but I doubt the world would become a drastically worse place.

But the second one? Hands down. Reading a lot probably comes second, but it’s not a close second (I think writing a lot would not be nearly so useful if I didn’t also read a lot, but I still think that if I had to choose between doing only one of writing and reading, writing is the more beneficial one).

It feels like I’ve got this magic make everything better tool which no one else has cottoned on to, and that’s a shame. So that’s the main reason you should write more: Not so that I (or anyone else) can hear what you have to say, but because it will make your life better.

Why will writing make your life better?

There are a lot of reasons, but here are what I think are the big ones (in order of least to most important):

Writing makes you better at writing

This is obvious, right? You practice something and you get better at it.

But why should you care? Getting better at writing is only important if you actually care about writing.

It’s because writing is not just something you do for yourself: It’s an incredibly valuable professional skill.

People make a big deal these days about how computer skills are important for every job, but it seems much less commonly observed that writing skills are important for every job. An extremely large proportion of business communication is written: Think how much time you spend in email.

Wouldn’t it be nice to be better at that?

And it’s not just nice: it’s super useful. Consider how many unclear or incoherent emails you’ve been on the receiving end of. Wouldn’t you like it if those were clearer, better thought out and generally held to a higher quality of writing. I don’t mean spelling and grammar here. I’m talking about conveying the point in a way that is easy to understand.

Your coworkers would like that too. By raising the quality of your writing you will make their lives better, and if making their lives better isn’t enough motivation for you, note that by making their lives better you are making them more likely to respond usefully to the email you’ve sent them (and making them like you more so more likely to respond usefully to you in general).

You can and probably do get better at writing email just by writing email, but anecdotally at least this seems not to happen all that much. I think it’s a mix of people not thinking of email in those terms and not getting the right feedback from it. Fortunately, by learning to write in other contexts you can just transplant those skills wholesale to when you’re writing email.

It’s also useful for the rare occasions where you need to really craft that perfect email. I’ve sent a couple work emails in my life where people who have read them have basically gone “This is an amazing piece. Can I forward this to several people I know who should read it?”. This is a good reaction to have (though I must admit these emails haven’t been super-effective, but that’s mostly because they’ve been “Here are these structural problems with the company” and the structural problems with the company have prevented the email from being acted upon. Writing is useful but not always enough).

So,  yeah. By writing more you will make your work life better.

Writing helps you learn things

One of the single best ways to learn something is to explain it to other people. Writing gives you an opportunity to do that: By writing about something you are forced to think of it in a way that you can explain to other people, which forces you to structure it in a way that you can understand.

I saw a great dialogue on twittter recently (I forget who between I’m afraid) which went something to the tune of “I don’t understand half the thing she writes about” to which the author in question responded “Confession: I don’t understand half the things I write about until I’ve written about them”. This also corresponds to my experience: Quite a lot of my blog posts are me writing about things so that I can understand them.

There is in fact an entire book on this subject (“Writing to Learn” by William Zinsser) which is still on my to read pile, so I won’t say too much more about this for now.

Writing teaches you to think

A piece of feedback I often get from people is that I think things through pretty thoroughly.

This is not a natural trait for me.

Oh, I naturally think about things a lot, but my native thought process is “Here’s a thing and here’s a thing and oh have you thought about that other thing and hey kittens are cute oh god why are things so terrible what were we thinking about again oh yeah stuff which is related to this other stuff is…” etc you get the idea. It’s not exactly thorough so much as scattershot.

The mechanism by which I turn this chaos into thinking things through thoroughly is that I write blog posts about it.

I don’t necessarily publish blog posts about it, but a lot of my blogging happens inside my head – I draft posts by teasing out a chain of thought and forcing structure onto it until something that starts to resemble a blog post emerges which I then later turn into a post. You can do this and then not actually sit down and write the blog post, and it’s still just as useful for structuring your thoughts, butit’s very hard to learn to do this without occasionally actually sitting down and writing the blog post.

This discipline of learning to structure my thoughts has been amazingly helpful. If this is something you have difficulty with or would like to improve, you should write more because it will literally make you better at thinking.

But I can’t write more because…

So that’s a bit about why to write more. But you can’t, because reasons. I think most of those reasons are excuses, and where they’re not excuses they are surmountable. Here are some common ones.

I don’t have the time

You probably do. I mean, some people genuinely don’t have the time, but for the most part people don’t find time they make time and “I don’t have the time” means “I’m not making this a priority”. Somehow every november hundreds of thousands of people find the time to write 50,000 words (note: I blog a lot and 50,000 words is more than my annual output).

Try setting aside a half hour block a week to write. See how it goes. You won’t write a lot in that time, but it’s a good point to start from. If it turns out to be easy to find that time, see if you can find some more time.

I think it’s also worth noting that you don’t actually have to be fully awake to write. Think about the thing you want to write over the course of your day, then either when you get home and want to veg out or in the morning before work or whenever you can just splurge out some words. A bunch of my blog posts are written in this state and so far no-one has called me on the difference (this may not reflect well on the rest of my blogging).

I don’t have anywhere to write

There are lots of free blogging platforms. Try tumblr, or medium, or wordpress.com.

If you really don’t want to write in public, fire up a text editor or MS word and write in there. Or fire up your email client and send yourself an email.

If you want to write with an audience but not in public, maybe try Facebook and see if it annoys your friends too much. Or I hear livejournal is still a thing, and it has good privacy controls for this.

No one would want to read what I write

I think you’re probably wrong, and what you have to say is likely to be more interesting than you think it is. I’ve found I’m a terrible judge of which of my posts people will actually interest people.

But if you’re not wrong, that’s OK. Almost none of the above relies on anyone reading it. The process of external feedback is helpful, but it’s in no way essential. You should start with the principle that you’re writing for yourself and other people are welcome to come along for the ride if they feel like it.

I’m terrible at writing

If only there were some sort of way to get better at that…

It’s dangerous for me to have a public presence due to race/gender/sexuality/etc.

This is a completely legit concern unfortunately. It’s also one I have no personal experience with dealing with, so I’m hesitant to offer too much advice on this one. If you’re comfortable doing so, try writing in public with a pseudonym. If not, I’d strongly encourage you to still try writing even if you feel you have to do it in private. Other people may have better advice than me here.

I don’t know what to write

So I’ve convinced you that you should write, and I’ve convinced you that you can write.

What do you do now? How do you decide what to write about?

Here are some suggested starting points. Pick one. If you can’t decide on one amongst several, pick the easiest one.

  • This is a thing that happened to me recently that was amusing.
  • This is a thing that happened to me recently that was annoying.
  • I disagreed with someone over a thing. Here is a persuasive piece for my side of the disagreement.
  • Here is a thing people often fail to understand.
  • Here is a thing people do that is annoying.
  • This is a thing people new to my job often get wrong.
  • This is a thing people who interact with people who do my job often get wrong.
  • This is a thing I am trying to learn
  • This is a review of a book I read recently
  • This is a review of a film I watched recently
  • Here is a cool thing about the place I live
  • Here is a thing that makes my life better when people do it
  • Here is a thing that makes my life worse when people do it
  • This is a thing I know that you might not have heard about
  • This is a thing I learned recently that I was surprised I had not heard about

If none of those grab you, allow me to offer my services as a muse: If you still can’t think of anything to write, email me. I’ll try to help you out.

Also, if you do write more as a result of this, I’m happy to offer my services as a publicist and tweet about it if you want people to read what you’ve written (things that are obviously spam need not apply, but I’ll otherwise do this even if I don’t agree with what you’ve written).

So, go and write something already.

This entry was posted in Writing on by .

Why mathematics makes you better at programming (and so does everything else)

There’s been a bunch of discussion on whether mathematics is programming, whether you need to learn mathematics to program, etc. on Twitter recently. Some of it has been quite heated. So naturally I thought I’d wade right into the hornet’s nest because I’ve got no sense of self-preservation.

There are essentially 4 levels of question going on here:

  1. Is programming basically mathematics?
  2. Do you need mathematics to be a good programmer?
  3. Is there a useful mathematics of programming?
  4. Can learning mathematics make you a better programmer?

The answers to these questions are respectively no, no, yes but you probably don’t care most of the time and yes.

Cool, well, that was easy. Are we done?

Not just yet. I’m going to spend some time unpacking the last of these because it’s the most interesting: Does learning mathematics make you a better programmer?

Well, yes. Of course it does.

So does learning to write. So does learning to cook. So does learning about philosophy of ethics, or about feminism. Learning another human language almost certainly helps you to be a better programmer. Your programming will probably improve if you learn Judo, or knitting, or tap dancing.

I’d go as far as to say that it is unusual to find a pair of skills where learning one will not in some way improve your ability in the other.

Part of this is just that whenever you learn a thing, you’re also learning about learning. You learn what works for you and what doesn’t, and the next time you need to learn something you’ll be that much better at it.

But there’s more to it than that, right? Surely mathematics teaches you more about how to program than just about how to learn things.

Well, yes, but it’s not because by learning mathematics you’re necessarily learning anything directly applicable to programming. Sure, you might be, and you might even be doing the sort of programming to which that maths is directly applicable. But if even if you’re not, learning maths will still make you a better programmer than say, reading Derrida (probably. I haven’t read Derrida so I might be wrong) or learning to cook (Almost certainly. I have learned to cook, and mathematics was way more useful. Your brain might work differently than mine).

Why?

Well, in much the same way that there is a sort of generalised skill of learning that you can improve just by learning things, there are all sorts of other generalised skills. For want of a better word (if you have a better word, do tell) I call them “microskills”. They’re mostly not things that people think of as skills – it’s things like breaking a problem down into parts, abstract thinking, learning to ask the right questions, perseverance when stuck, learning when not to persevere when stuck, and thousands upon thousands of others.

People tend not to think of these as skills because they think of them as innate abilities. Stuff you’re just good at or bad at and you’ve got to cope with that. And I mean, it’s certainly possible that you can be innately good or bad at these (and I may be foolish enough to touch this debate but I’m not touching that one with a barge pole), but regardless of innate ability they all share a characteristic that makes them skills: they get better if you practice them.

Almost every skill you learn will depend on some of these microskills. Which ones is not at all fixed: People compensate by trading off one for the other all the time. You might be less good at working memory, so you practice your organisation skills and keep meticulous notes. You might be easily distractable and lack the ability to methodically push through a problem but make up for it by creative flashes of insight (yes you can learn to have creative flashes of insight, no creativity is not a fixed thing that you can never get any better at).

But it is at least biased. Different skills will encourage you to draw on different microskills. Mathematics will draw heavily on analytical ones – abstract problem solving, hypothesis generation, breaking down a problem into smaller parts, etc. All of these are things that are super useful in programming. So by learning mathematics you’ve developed all these microskills that will later come in super handy while programming.

But, of course, you can develop the same microskills when programming, right?

Wellll….

OK, no. I’m going to go with yes. You will absolutely develop all of the relevant microskills in programming. Learning to be good at programming by doing programming is 100% a viable strategy and lots of people successfully follow it.

The thing is, you won’t necessarily develop them to the same degree. Some you will develop more, some you will develop less. And this isn’t necessarily because the ones you would develop less in programming wouldn’t be useful to be better at.

There is essentially one key thing required for learning to get better at something (HORRIBLE OVERSIMPLIFICATION ALERT): Feedback

Feedback is how clearly and strongly you discover whether you did well or badly at a thing. If you can just try something and immediately find out if it worked or failed, that’s great feedback. If you try something and 3 months later you get a positive result that might be the result of that something (or it might be the result of something else that happened in the intervening 3 months), well… not so much.

Feedback is essentially how you get better because it lets you know that you are getting better, and that’s how you direct your learning. As you get better at a thing, the amount of feedback you tend to get degrades - and this isn’t necessarily because the benefit you’re getting from improving is decreasing (though it often does) it’s often because the benefit is moving further down the line, or into things that are harder to quantify (for example, the benefits of “a sense of good taste” in programming beyond a basic avoidance of complete spaghetti code are often not immediately apparent until you’ve sat atop a mountain of technical debt).

And this is where learning other skills can be super useful, because they provide different feedback loops. Sometimes they even let you develop microskills that appeared completely irrelevant and then once you had them turned out to be super useful (silly example: Strength training to develop core strength. Totally inapplicable to programming, right? Well, yeah, except that it turns out that actually not being distracted by lower back pain while you program is quite handy), but most of the time it’s just that the feedback cycle on them is different and the incentive structure for developing that skill is different – mathematics will lean more on some microskills than programming does and less on others. Similarly it will provide better feedback for some, worse for others.

I could provide some specific examples of which things I think mathematics will help you develop better, but I won’t. It’s very hard to judge these things, and people can argue endlessly about which is better for what. But the specifics don’t matter: Basically all forms of abstract thinking are useful for programming, and the only way that we could reasonably believe that mathematics and programming could provide exactly the same set of feedback loops and incentive structures as each other would be if we believed that they were the same thing.

And that would be silly.

This entry was posted in Numbers are hard, programming on by .

A system for coordinating groups

Having flatmates again means that there are now a whole bunch of things that are a shared responsibility that were previously all down to me. This has caused me to think about the problem of how you coordinate such things.

At the moment we’re adopting a really boring approach: We’re talking about things and doing what seems to make sense.

Obviously this sort of natural human interaction is not to be tolerated and must be replaced with PRECISE ALGORITHMIC RITUALS and EFFICIENT MARKET BASED SOLUTIONS.

Err. Um. Yes.

That is to say, an interesting approach to occurred to me a while ago and I concluded it was an obviously bad idea. But it keeps coming back to mind and I keep solving the problems that make it an obviously bad idea to the point where now it’s only a probably bad idea, and it might be an interesting one.

I haven’t tried it, and for the problem of flatmate decision making I’m not sure I’m that interested in trying it. At only 3 people (and two cats, but the cats don’t really pull their weight, the tiny furry slackers) talking about things mostly works as a solution. For larger groups this might be a more compelling approach.

The basic setup is this: You have a number of people, and a number of things they need to share. For example, for flatmates it might be “doing the cooking”, “cleaning the kitchen”, “cleaning the bathroom”, etc. You want to divvy these up in a way that is not only fair but also matches individual preferences: For example, someone might be strongly averse to cleaning the bathroom but perfectly happy to do a disproportionate share of the kitchen cleaning to make up for it, and everyone might much prefer to cook than to clean. On the other fairness dictates people do about an equal amount of “the work”, whatever that abstraction might be (time is not necessarily a good measure. Cleaning the bathroom might be a short task that some people would trade much more time on other tasks to do).

So how do you accommodate these constraints?

The system works as follows: You have a number of currencies. These should correspond relatively closely to the tasks you want to share. For example they could be “cooking a meal”, “cleaning the bathroom”, “minutes spent cleaning the kitchen” (because the latter is a more unbounded task, depending on the amount of mess made by cooking, while the former are more defined tasks). They don’t have to be quite so closely tied to the tasks – you could e.g. define a bundle currency which is just “total amount of time spent cleaning” but it’s probably better if they are. You do however need to ensure that every action only corresponds to one currency.

Everyone has a balance in each of these currencies, starting at zero.

Balance is accrued at an agreed upon rate by performing actions. You cook a meal, you get one “cook a meal” credit, you spend half an hour cleaning the kitchen you get 30 “clean the kitchen” credits, etc. When this happens, everyone else who is a beneficiary of that action (this is agreed upon per action type. e.g. cooking a meal the beneficiaries are just those who eat it, cleaning the kitchen the beneficiaries are the people who made the mess in it, cleaning the bathroom the beneficiaries are everyone) acquires an equal share of debt totalling that credit. So e.g. if I cook a meal for myself and one other person, I acquire 1 meal-unit of balance and they lose 1 meal-unit of balance. If I cook a meal for myself and two other people they each lose half a meal unit of balance.

Note that:

  1. All currency units are infinitely divisible even though it might not be possible to perform fractional actions to get them (so you can have fractional balances in cooking meals even though you can’t cook a fraction of a meal).
  2. Every operation is zero sum and always leaves the total balance at zero.

When it comes time to perform an action, the person available to perform it who has the lowest balance in the relevant currency always performs it. If there are multiple people with the lowest balance, then they draw straws or some other random way of deciding.

Once you’ve decided who should perform the action, they may choose instead decide to conduct an auction. This consists of saying “I need X units of this currency and am willing to exchange it for this other currency”.  For example I could say “I need two units of bathroom cleaning and am willing to exchange it for cooking credits”.

Contraints:

  1. They need to ask for enough of the relevant currency so that they would no longer have the lowest balance (but can ask for any amount more than that if they choose).
  2. They cannot exchange it for any currency they are currently in negative balance for.
  3. There must be at least one person who has that amount of balance available (otherwise there isn’t anyone who can bid in the resulting auction)

Once they have made this offer, a sealed second price auction occurs amongst all the people who have enough balance in the asked for currency. Additionally, the auctioneer also puts up a bid which effectively acts as a reserve price. This reserve may not be greater than their balance in the offered currency (so if I’ve a balance of 5 extra meals cooked, my reserve can’t be greater than 5, but it may be less). People may choose not to bid, but they should keep this secret as well (it’s effectively the same as bidding more than the upper bound on the reserve price).

The winner of the auction is the person who bid the lowest (because this auction is backwards from normal second price auctions. They are giving the thing being auctioned, not receiving it). If that’s the auctioneer with their reserve, oh well they have to perform the action after all. If not, they give the auctioneer the asked for amount of balance (i.e. it is subtracted from their balance and added to the auctioneer’s balance) and then they receive amount of the offered currency equal to the second lowest bid (this may of course be the auctioneer’s reserve). In the event of a tie, the winner is a randomly selected person with the lowest bid who is not the auctioneer, and the second lowest bid is the same as that tied lowest bid.

Now you repeat the process. If the person with the new lowest balance is willing to perform the action, they do so. If not, they may offer another auction. This continues until someone is either willing to do it (or doesn’t have anything in credit they can auction off in exchange) or an auction fails to meet its reserve.

I haven’t done any proofs of fairness here, and to be honest it’s probably not all that fair, but it should have the property of allowing people to exchange chores at a mostly reasonable rate. It almost certainly will be vulnerable to arbitrage and other financial shenanigans in practice, but the fact that you can’t really use money to make more money and acquire debt from other people contributing to the system should limit the worst of the excesses of market based systems.

(I may do a simulation of this at some point to see what happens. I got about half way through writing one and then couldn’t be bothered to continue, but that might change).

This entry was posted in Uncategorized on by .

Playing beeminder on hard mode by adding backpressure

So as you might have gathered I really like beeminder. A proper advocacy post will follow at some point, but for now this is just a report on a new experiment I’m going to be performing with it.

One of the things I don’t like about beeminder is the eep days. That is, days where you’re about to fall off and need to do some panicked activity to prevent you from doing so. In principle they’re fine – they provide good motivation and remind you of your duties as a good anti-procrastinator – but there are two main problems I have with them:

  1. They tend to bunch together. Generally if I’m getting eep days it’s because I’m falling behind on ability to meet my goals, usually do to energy or time constraints, so I can get several goals eeping at once or in short succession.
  2. They tend to reoccur. The problem is that goals are necessarily set at a reasonable daily rate, so necessarily if I just do a reasonable daily rate on an eep day it will reoccur tomorrow.

It should be relatively easy to prevent eep days from occurring. You just work a bit extra when you have the energy to do so to build up a buffer, then resume a normal rate and you’ve got this nice comfort zone.

Of course, it doesn’t work out that way. I naturally procrastinate until things become pressing and the buffer gradually dwindles away and the eep days happen.

Hmm. What I need is some sort of tool to stop procrastinatingSay, a way to be reminded of it and penalised for it.

Do you know of such a thing?

Oh, right. There’s this thing called beeminder. I could beemind my beeminding.

So I wrote a simple script to add a smooth backpressure to beeminder. The way it works is that I have a “Do less” goal (I’m really enjoying the do less goals) which tracks how many goals I have with less than a week left till I fall off the road each day. This goal is set to allow me 7 due-goal-days a week, so I can safely have one goal which is approaching due but if I have two then I will exceed my allowed rate and rapidly head towards failing my pledge.

I currently have two goals which are coming due (read more non-fiction and go to the gym more), so I’d better hurry up and get one or both of those sorted if I don’t want to derail.

Edit: Note that I’ve changed this to only apply to “hustler” (do more) type goals, as I found it didn’t work well with “do less” goals.

This entry was posted in Beeminder on by .