Category Archives: Uncategorized

2015 in review

I don’t think I’ve done these in previous years, but they seem to be a thing this year and according to todoist I’m due a blog post (overdue by 3 days actually), so here we go.

So… 2015. That was a weird year, huh?

This year I have:

  1. Moved country (again)
  2. Upgraded my internet celebrity status from D-list to C minus-list if you’re in specific niches
  3. Earned precisely £0 (note: This is a slight technicality).

What happened?

Well, if you may recall, in 2014 I moved to Zurich to work for Google. Despite what people thought from the timing of the post, this was not an April fools joke but an actual thing.

It didn’t really work out and I lasted about 6 months. For a variety of reasons, Google was not a good environment for me, and between that and it hampering my ability to work on my own stuff in my spare time, I got a lovely inside view of what happens when the sub-clinical depression I’ve been mostly coping with all my life removes the ‘sub-‘ from its name (OK, I didn’t actually get diagnosed, because I figured out how to self-medicate through a judicious application of quitting my job instead).

So, starting the new year unemployed was pretty great, even if I was still in one of the most expensive cities in the world.

I’d intended to work on a variety of projects, and actually spent the initial part of the year brushing up on combinatorial optimisation and learning about integer programming. But I had this weird little project called Hypothesis that I’d written back in 2013 to learn Python, and for some weird reason people had started using it, so I thought I might fix a few bugs, maybe work on some features, etc.

So, um, yeah.

For me 2015 has basically been the year of Hypothesis. This was very much not intended from the outset. I originally intended to put in a couple months work on it. I even made a concerted decision to give it up. Now I’m building a business around it.

It’s been an interesting experience for a number of reasons. It involves research to solve practical problems, randomized algorithms, combinatorial optimization, API design, coming up with good abstractions, and software quality. All it needs is voting to become my ideal project.

…voting and financial viability that is. Which is why I’m trying to build a business around it.

The business side of things is going OK. I’ve had some work this year (which I have not yet paid myself for from the business’s bank account, which is why £0 earned in 2015), and a bunch more scheduled for the new year. If I can keep up my average income over the next three months for the rest of 2016, I’ve got something pretty sustainable. I don’t yet know if I can, but I’m a lot more hopeful about the possibility than I was 3 or 4 months ago.

This has also got me to go to a lot more conferences and give talks, which has been a rather great thing for me. Turns out getting up in front of a few hundred people and talking entertainingly is a thing I can do. I had no idea. I credit this blog with a lot of that.

Other things than Hypothesis have happened this year too…

Writing

Obviously I’ve written stuff this year. I always write stuff. Notable things I’ve written here on this blog include:

This was also the year I got into writing fan-fiction.

It started with Stargate Physics 101. While this is Stargate fan-fiction, it’s really a comedy about software testing and existential threats. “I send this to software developers to teach them about testing” is an actual thing I have been told about this piece. “My job is a lot like this, but with fewer genderqueer werewolves” is an actual thing I have said about this piece.

I’ve also since been writing The Rules of Wishing. This is fan-fiction for Disney’s Aladdin.

I’ve also started writing docs.drmaciver.com, which is facetiously described as a handover manual for being DRMacIver but is mostly a catalogue of stuff right now.

Living situation

For the first time (not counting holidays) in more than a decade, I’ve been living with my parents since May. It’s been an interesting experience.

The living with my parents part has been fine actually. We get along fairly well, even if I occasionally start arguments about shower lengths and ham sandwiches. The real problem with living with them is that they live in the country, and the country is bullshit.

One result of living in the country is that I’m doing much less walking (if this sounds backwards to you, I assure you it is not. Living in the city naturally integrates a lot of walking into your life. Living in the country mostly surrounds you with roads that are not safe to walk on and would take you an hour to get anywhere), which means that my overall fitness level is way down. I’ve been actively trying to combat that in the last month or so, which seems to be having some effect.

Another consequence of living here is that it really helps me grind my driving skill. I’ve probably driven about an order of magnitude more total hours in the last 6 months than in the rest of my life put together.

This has both been interesting and frustrating. I’m still politically quite against cars, and I don’t really like being so dependent on them. However, it’s taken my driving skill from “I don’t think you should trust me behind the wheel of a car” to being pretty OK.

Beeminding

I went from wildly enthusing about beeminder to just quietly using it. It petered out for a while, but now is picking back up again.

My two main things for beeminding are todoist and exercise. I’ve also got some tagtime goals for work and reading non-fiction.

What next?

My plans for 2016 are mostly “do what I’m doing now except more so”. I’m in the process of getting back in shape. I’d like to do that, but more so. I’m in the process of building a business. I’d like to do that, but more so.

I would definitely like to move back to civilization. I’m giving strong consideration to this being not London, but it will have to be somewhere London accessible.

Another thing that will happen in 2016 that I’m excited about is that 2016 will involve travel to at least two countries I’ve never been to before. I’m going to Namibia in January (and offering a great deal on Hypothesis training until I do!), and later in the year I’ll be going to Mexico for my brother’s wedding.

I tend not to travel much, so both should be interesting experiences.

Other than that, I have some interesting research directions I want to take Hypothesis/Conjecture in, and may be exploring some plans for how to focus on that. Watch this space.

 

This entry was posted in Uncategorized on by .

Mentors as a service

This is an idea I keep coming back to in my head. It’s possible it already exists and I just haven’t seen it, or that it’s a niche that’s sufficiently well served by things like coding bootcamps, etc. (though making them serve this niche is somewhat problematic).

I’m not quite motivated enough to try it myself because it doesn’t really play well to my strengths, but I thought it might be worth putting out there to see what people think.

The starting point:

Getting more junior developers into the industry is a good thing. You’ve got the generic problem that everyone struggles to hire good developers, the diversity of tech problem, and of course it’s great to be able to help people get a good job in a well paying industry.

But we’re also really bad at hiring junior developers. We’re bad at hiring in general, but the proxies we use for hiring break down especially badly for juniors. This makes hiring for the roles either even more of a time sink than hiring normally is or a complete crapshoot.

Then once we have hired junior developers we’re also bad at that. The problem is that junior developers require a lot of support. This takes up a lot of time from your more senior developers. They may also just not be very good at it – teaching is a specialised skill, and even developers who are otherwise good communicators may not be great at it. The result is that a lot of junior developers flounder because they are not properly supported.

This may not be your experience – some companies are much much better at hiring junior developers than others are – but it’s been mine, and I believe it to be sufficiently common that there’s a problem here to be solved.

And I think there’s a business to be made in solving it, and a business of the best sort: One where you can make money off helping everyone involved in the process be better off.

Both of these problems are for the most part generic:

  1. Junior developers are much more interchangeable in terms of their skills than senior developers, because the primary characteristic of a junior developer is ability to learn. You might want some basic sorting by aptitude and preferences – e.g. someone like me with the visual skills of a brick you probably shouldn’t sort into front-end (note: This almost happened to me), but for the most part there’s a fair bit of flexibility here.
  2. Most of the questions a junior developer will have are things like “What’s a list comprehension?” and “git wtf?!”. They’ll definitely have questions specific to your company, but so will intermediate and senior developers. The real difference with juniors is the absence of the basic skills.

Both of these seem like they don’t have to be solved by an individual company.

Imagine the following setup:

Our hypothetical company, JuniorDevCo, is constantly interviewing potential junior developers at a rate that is designed to give them a constant pool of candidates (say somewhere in the 10-50 range depending on hiring rate).

Someone can then come to them and say “We are looking to hire someone for $ROLE”. JuniorDevCo will vet the company, negotiate a salary for the role, and then suggest a couple of candidates (probably just the 5 who it’s been longest since they’ve been interviewed unless there are specific requirements) for the company to meet and pick amongst in the obligatory step to give the company a sense of agency.

So far what I have described is also known as “a recruitment agency”, or at least is close enough to one that it doesn’t seem particularly interesting.

The interesting bit is that when you hire these developers they come with an associated mentor, and access to JuniorDevCo’s internal help system (I am imagining this is something morally equivalent to “A slack channel with a bunch of people employed by JuniorDevCo on there to help + some sort of knowledge base with good documents to send them in response to specific questions). They meet with the mentor regularly to discuss how things are going and to get guidance, and they can message them for help at any time.

In exchange, JuniorDevCo gets a decent monthly fee, and a bonus if after one year the junior dev is still employed at the company (at which point the company can choose to either keep paying for them to have a mentor or not as they prefer).

Would this work?

I don’t know, but I think it could.

Some specific thoughts:

  1. It takes a very real pain point and says “Hey, so, this is costing you a lot of money, everyone hates it, and you’re not getting good results. If you pay us less money than it is costing you then we can make the pain go away and get you better results.”
  2. A process of constant hiring lets you really refine the process and see what works. It also helps a lot in analysis of things like bias – is your process decently gender neutral, does it have a racial bias, etc? These are the sort of things you can do if you’re hiring at a high rate and are lost in the noise if you’re not.
  3. Making the numbers work out in terms of how much you charge might be tricky, but you’re saving a lot of time from senior developers, so it should be viable to get a senior developer rate per mentor, and good tooling and a good knowledge base can help keep their workload manageable.

It seems weird that setups like this don’t already exist, which is probably a sign that I don’t know the right keyword to google for and they actually do already exist. It may also be that there is a fatal flaw in the plan that I’ve not yet seen.

Edit to add: Having said that this doesn’t play well to my strengths, of course parts of it do. In particular I’m thinking that the remote mentoring part is something that I both can do quite well and could easily offer as part of my general business. If this is something you think your company would be interested in, drop me an email.

This entry was posted in Uncategorized on by .

Notes on headache self-care

I have a history of headaches. Not full blown migraines (usually), but long nagging low-grade ones. It’s almost guaranteed that some of this is caused by spending too much time in front of the computer, but what am I going to do? Spend less time in front of the computer? Pfft.

I have a variety of self-care strategies and was talking to a friend about them earlier. It occurred to me that they might be useful to share more broadly.

Note: I’m not a doctor, and what professional experience I do have on this subject is out of date and was never very good in the first place. These are strategies that have very much been derived from my personal experience, and I can’t promise that all of them will help you or will be good ideas for your case. Some of them may be actively bad ideas for your case.

Dehydration

Accidentally getting dehydrated is the easiest way to get a headache. If you have a headache, the first thing you should do is drink a large glass of water. It won’t fix it immediately and this shouldn’t stop you from immediately going on to more heavy duty lines of attack if you’ve got a serious headache, but it will never hurt and will probably help at least a bit.

Drugs

My primary drugs of choice are ibuprofen and caffeine (usually as coffee). Caffeine is helpful even if your headache isn’t caused by caffeine withdrawal (Exceptions: Excess of caffeine can cause headaches. Also some people get caffeine induced migraines). Something something blood flow.

Ibuprofen, well, it’s a pain killer. It seems to work really well for me. My experience is that NSAIDS are very effective on me and paracetamol does nothing except for make me worry about my liver (it’s perfectly safe at sensible dosages, but I don’t like things where taking too much of them can kill you).

That being said, I take probably more ibuprofen than I should, and probably with less food in my stomach than I should. I don’t particularly recommend following my lead. Note also that if you’re doing this on a daily basis then this can cause headaches [Edit to add: This is true of all over the counter painkillers, not just NSAIDS]

Most of the time some combination of the above is enough to shake most headaches for me. When it doesn’t and I’m verging into migraine territory I like some combination of paracetamol and codeine (I know I said paracetamol doesn’t work on me, but it apparently has a synergistic reaction with codeine. Also you can’t get neat codeine, and the amount of codeine in an ibuprofen + codeine mix without that reaction is apparently basically a placebo). I like migraleve, but anything with paracetamol and codeine in it is probably a good idea. I usually deploy this on top of ibuprofen, which I am given to understand is perfectly safe.

If you are regularly getting headaches you cannot self-medicate with over the counter drugs then you should see a doctor and not consult some guy on the internet’s self care advice.

Tension headaches

Fun fact: I trained as a massage therapist.

Supplementary information for fun fact: I didn’t qualify (scheduling conflicts between exams for my massage course and my mathematics degree). Also, as a massage therapist I make an excellent software developer.

However, this does get me a bit ahead of the game in terms of self-care for tension headaches.

In theory if you’re being sensible you should try these techniques first. In practice I usually reach for the ibuprofen and either the problem goes away or is sufficiently severe that I’m in no mood to be sensible. However, these are particularly useful if you have persistent ongoing headaches that you think might be tension related. The NHS guidelines on identifying a tension headache are useful, but to be honest if you’re feeling any sort of muscular tension and experiencing a headache you might as well try these. It may not help the headache, but it will distract you for a bit, won’t hurt and will probably help the muscle tension.

Upper back and shoulders

These are really hard to self massage, because most of the things you can do to self massage will make the other side worse. I use stretches to deal with these. I used to have a “The original backnobber 2” (yes that’s its actual name), which looks like an elaborate sex toy and I found almost entirely useless. Some actual elaborate sex toys may be more useful here – e.g. the Hitachi magic wand is usually sold as a self-massage device – but I can’t say that my experience with vibration systems (from a massage chair. Stop snickering) has actually been very useful for reducing upper back and shoulder tension.

I do have some stretches that I do to help if I have tense shoulders:

  1. Lace your fingers together with your hands in front of you, palms facing outwards and push them forward as far as you can.
  2. Do the same thing but with your hands behind you and your palms facing towards your body, then bend at the waist 90 degrees so that your hands are straight up into the air and stretch backwards.
  3. Do ridiculous exaggerated shoulder rolls
  4. Twist your upper body from side to side around a vertical axis

If these don’t help, I recommend an actual back massage. Get a friend or partner to give you a back rub (I don’t currently have any good recommended tutorials for that, sorry) or go to a professional (which is advice I don’t follow enough myself).

One technique that does work for me reasonably well is as follows:

  1. With both hands at the same time, dig your fingers in between your shoulder blades and your spine (never put pressure on the spine! It’s the muscles next to it). Your elbows should be as high up as they would go.
  2. Gradually lower your elbows, pulling your fingers along with them, until they come off the top of your shoulders.

Can’t promise that one helps, but it feels nice at least.

Neck and scalp

These are fortunately much easier to self-massage.

I have three main techniques here:

  1. Temple rub: You’re probably already doing this one. Take your three main fingers and place them on your temples. Press reasonably firmly and move the skin in circles, from front to back.
  2. Scalp massage: This is much easier if you’re not me and still have hair. Grab thick bunches of it near the roots and use it to move your scalp about. Similar to the temple rub, do it with both hands at once, rotating from front to back. I usually do this once on top and once at the back of my skull near the top. If, like me, you lack hair, you can do something similar by using the base of your hands, pressing firmly into the skin and using friction to move the scalp.
  3. Massage the side of the spine from the base of the necks to where it joins your spine. Do this by placing the tips of your fingers there, digging in and pulling outwards. Then move up slightly and repeat the process until you run out of spine.

You can also do neck streches by slowly rotating your head through a head shake as far as it can go in each direction, and also by bending your neck from side to side.

These don’t always help, and they don’t usually completely get rid of a tension headache for me, but they often reduce it to a level where a previously intolerable headache becomes tolerable and much more manageable with painkillers.

Sleep

Sleep is also the cause of a lot of my headaches, but the worst ones usually go away after I’ve slept. If a headache is rendering me useless, sometimes the best solution is to deploy the above solutions (sans caffeine) and go to bed early and hope it will be better in the morning. It usually is.

This entry was posted in Uncategorized on by .

Having another go at a vanity IRC channel

A while ago I experimented with a vanity IRC channel: #DRMacIver on Freenode. It kinda petered out, but I thought it might be nice to try again, so I’ve resurrected it.

So do join. If you’re not familiar with IRC, join anyway! Here’s a link to join via IRCCloud (don’t worry, it’s free).

Channel rules:

  1. Hypothesis code of conduct is in effect.
  2. There is no concept of off-topic discussion. You can talk about whatever you like. However note that I will be talking a lot about what I’m doing, posting links to my stuff, etc.

I’m basically treating it as an extended semi off the record commenting system for this blog and for other places I post things on the internet, so you’ll also get links to my fanfic, things I need help with, etc. I’ll probably set up some github hooks so most of my projects send commits in there.

This entry was posted in Uncategorized on by .

Mergeable compressed lists of integers

Alexander Shorin’s work on more configurable unicode generation in Hypothesis has to do some interesting slicing of ranges of unicode categories. Doing both generation and shrinking in particular either required two distinct representations of the data or something clever. Fortunately I’d previously figured out the details of the sort of data structure that would let you do the clever thing a while ago and it was just a matter of putting the pieces together.

The result is an interesting purely functional data structure based on Okasaki and Gill’s “Fast Mergeable Integer Maps”. I’m not totally sure we’ll end up using it, but the data structure is still interesting in its own right.

The original data structure, which is the basis of Data.IntMap in Haskell, is essentially a patricia trie treating fixed size machine words as strings of 0s and 1s (effectively a crit-bit trie). It’s used for implementing immutable mappings of integers with fast operations on them (O(log(n)) insert, good expected complexity on union).

With some small twists on the data structure you can do some interesting things with it.

  1. Ditch the values (i.e. we’re just representing sets)
  2. Instead of tips being a single key, tips are a range of keys start <= x < end.
  3. Split nodes are annotated with their size and the smallest interval [start, end) containing them.

When using this to represent sets of unicode letters this is extremely helpful – most of the time what we’re doing is we’re just removing one or two categories, or restricting the range, which results in a relatively small number of intervals covering a very large number of codepoints.

Let T be the number of intervals and W the word size. The data structure has the following nice properties:

  1. Getting the size of a set is O(1) (because everything is size annotated or can have its size calculated with a single arithmetic operation)
  2. Indexing to an element in sorted order is O(log(T)) because you can use the size annotation of nodes to index directly into it – when indexing a split node, check the size of the left and right subtrees and choose which one to recurse to.
  3. The tree can be automatically collapse tointervals in many cases, because a split node is equivalent to an interval if end = start + size, which is a cheap O(1) check
  4. Boolean operations are generally O(min(W, T)), like with the standard IntSet (except with intervals instead of values)
  5. Range restriction is O(log(T)).

Note that it isn’t necessarily the case that a tree with intervals [x, y) and [y, z) in it will compress this into the interval [x, z) because their common parent might be further up the tree.

An extension I have considered but not implemented is that you could potentially store very small subtrees as arrays in order to flatten it out and reduce indirection.

In particular the efficient indexing is very useful for both simplification and generation, and the fact that merging efficiently is possible means that we can keep two representations around: One for each permitted category (which helps give a better distribution when generating) and one for the full range (which makes it much easier to simplify appropriately).

Here is an implementation in Python. It’s not as fast as I’d like, but it’s not unreasonably slow. A C implementation would probably be a nice thing to have and is not too difficult to do (no, really. I’ve actually got a C implementation of something similar lying around), but wouldn’t actually be useful for the use case of inclusion in Hypothesis because I don’t want to add a C dependency to Hypothesis just for this.

 

This entry was posted in Code, programming, Python, Uncategorized on by .