David R. MacIver

Me

I'm a software developer with a strong background in mathematics. My current focus is on Python and testing, but I've a history of being involved in diverse areas of back-end work in a wide variety of languages. Historically I've focused particularly on building, developing and maintaining data-centric production systems.

Employment

Self-employed Open Source developer and consultant (January 2015 to Present)

I took a sabbatical to work on an open source project of mine and then turned it into a business.

Hypothesis brings advanced testing techniques to Python. It's originally based on the Haskell library, Quickcheck, but has many innovations on the core idea that make it one of the best libraries of its kind in any language, and certainly the most suitable for use in an imperative language.

The project has become quite successful, and I've been spending some time doing consulting and training on its use and paid custom development on it.

Senior Software Engineer at Google (June 2014 to December 2014)

I worked on the system responsible for integrating data from the web into Knowledge Graph, Google's structured knowledge base. I worked on monitoring, integrating different data sources, did some extensive refactoring of the pipelines, and experimented with algorithmic improvements. Unfortunately, Google turned out to not be a good environment for me, so I decided to leave fairly quickly.

Backend Software Developer at Lumi (April 2013 to April 2014)

I primarily worked on the recommendation engine at Lumi, but also spent a lot of time on the performance and stability of the rest of the backend infrastructure. In my recommendations work I mostly focused on issues of recommender diversity - how to give a constant stream of recommendations you like without overwhelming you with any one subject that's having a busy day.

Senior Software Developer at Aframe (February 2010 to March 2013)

I started out at Aframe two days a week working on some maths for them - primarily centered around recommendation - but through a distressing inability to conceal competence ended up working on more and more areas and acquiring more and more responsibilities - first I ended up working on upgrading the feature set of one of the existing RabbitMQ ruby clients, then I ended up answering everyone's questions about everything, then I ended up taking over various parts of the system that needed improvement. I authored or at least interfered with most of the back end code and grudgingly conceded that if anyone counted as the system architect it was probably me. Additionally there was a period where I masqueraded as head of development and I was very involved with hiring and mentoring new back-end developers.

Engineer for LShift working on RabbitMQ (December 2009 to March 2010)

I mostly worked on the client libraries: primarily the Java one, but also the Erlang and C# ones. I worked on cleaning up the API, fixing bugs and improving performance. I did occasionally touch the server code, but didn't do very much with it.

Statistical Contracting for Wordtracker (June 2009 to November 2009)

Just a bit of part time work helping Wordtracker analyze their data about search - investigating user search behaviours and the quality of different metrics.

Engineer and Researcher for Trampoline Systems (August 2007 to September 2009)

My most important role at Trampoline was as the primary researcher and implementer of SONAR, which was software for doing natural language processing and social network analysis on corporate email and document systems. I made the system simpler to understand, produce higher quality results and scale more effectively. I learned a lot working on it, both in terms of specific theoretical tools and the practical side of building such systems (such as when not to trust computer science researchers claims...).

Prior to that I worked on our social network visualisation software. We developed Metascope, a tool for ONA consultants to help them better work with their survey data, in tandem with a partner company. They report that it has simplified their work flow immensely. We then built SONAR Expertise, which takes the ideas and technology of Metascope and integrates them with SONAR to provide a high level visualisation of a company's social structure.

Engineer for Softwire (February 2006 to July 2007)

This was my first job in software. While there, I worked with others on several projects for different clients. I picked things up pretty quickly, initially starting with front end work then moving into a mix of back end and front end work as I learned more. The laundry list of tech is fairly standard stuff for a Java shop: Spring, Hibernate, iBatis, Oracle, JBoss, etc. I always worked in teams while there and was often responsible for helping out new starters on a team. By the end of it I had been given responsibility over several key parts of projects, including the statistics reporting subsystem for an ePublishing site, the security model for the same and a UI redesign for an internal data inventory system for a public safety provider.

Education

Pembroke College, University of Cambridge (2001 - 2005)

I did my degree in mathematics. I received Class I Honours in my BA and Merit in my Part III (equivalent to an MMath)

Writing

I write extensively on a variety of subjects, mostly on my blog, which has a fairly respectable base of readers. Notable technical blog posts include:

Other interesting things I have written include:

Fun Facts