David R. MacIver
Me
I'm a software developer with a strong background in mathematics. My current focus is on Python and testing, but I've a history of being involved in diverse areas of back-end work in a wide variety of languages. Historically I've focused particularly on building, developing and maintaining data-centric production systems.
Employment
Self-employed Open Source developer and consultant (January 2015 to Present)
I took a sabbatical to work on an open source project of mine and then turned it into a business.
Hypothesis brings advanced testing techniques to Python. It's originally based on the Haskell library, Quickcheck, but has many innovations on the core idea that make it one of the best libraries of its kind in any language, and certainly the most suitable for use in an imperative language.
The project has become quite successful, and I've been spending some time doing consulting and training on its use and paid custom development on it.
Senior Software Engineer at Google (June 2014 to December 2014)
I worked on the system responsible for integrating data from the web into Knowledge Graph, Google's structured knowledge base. I worked on monitoring, integrating different data sources, did some extensive refactoring of the pipelines, and experimented with algorithmic improvements. Unfortunately, Google turned out to not be a good environment for me, so I decided to leave fairly quickly.
Backend Software Developer at Lumi (April 2013 to April 2014)
I primarily worked on the recommendation engine at Lumi, but also spent a lot of time on the performance and stability of the rest of the backend infrastructure. In my recommendations work I mostly focused on issues of recommender diversity - how to give a constant stream of recommendations you like without overwhelming you with any one subject that's having a busy day.
Senior Software Developer at Aframe (February 2010 to March 2013)
I started out at Aframe two days a week working on some maths for them - primarily centered around recommendation - but through a distressing inability to conceal competence ended up working on more and more areas and acquiring more and more responsibilities - first I ended up working on upgrading the feature set of one of the existing RabbitMQ ruby clients, then I ended up answering everyone's questions about everything, then I ended up taking over various parts of the system that needed improvement. I authored or at least interfered with most of the back end code and grudgingly conceded that if anyone counted as the system architect it was probably me. Additionally there was a period where I masqueraded as head of development and I was very involved with hiring and mentoring new back-end developers.
Engineer for LShift working on RabbitMQ (December 2009 to March 2010)
I mostly worked on the client libraries: primarily the Java one, but also the Erlang and C# ones. I worked on cleaning up the API, fixing bugs and improving performance. I did occasionally touch the server code, but didn't do very much with it.
Statistical Contracting for Wordtracker (June 2009 to November 2009)
Just a bit of part time work helping Wordtracker analyze their data about search - investigating user search behaviours and the quality of different metrics.
Engineer and Researcher for Trampoline Systems (August 2007 to September 2009)
My most important role at Trampoline was as the primary researcher and implementer of SONAR, which was software for doing natural language processing and social network analysis on corporate email and document systems. I made the system simpler to understand, produce higher quality results and scale more effectively. I learned a lot working on it, both in terms of specific theoretical tools and the practical side of building such systems (such as when not to trust computer science researchers claims...).
Prior to that I worked on our social network visualisation software. We developed Metascope, a tool for ONA consultants to help them better work with their survey data, in tandem with a partner company. They report that it has simplified their work flow immensely. We then built SONAR Expertise, which takes the ideas and technology of Metascope and integrates them with SONAR to provide a high level visualisation of a company's social structure.
Engineer for Softwire (February 2006 to July 2007)
This was my first job in software. While there, I worked with others on several projects for different clients. I picked things up pretty quickly, initially starting with front end work then moving into a mix of back end and front end work as I learned more. The laundry list of tech is fairly standard stuff for a Java shop: Spring, Hibernate, iBatis, Oracle, JBoss, etc. I always worked in teams while there and was often responsible for helping out new starters on a team. By the end of it I had been given responsibility over several key parts of projects, including the statistics reporting subsystem for an ePublishing site, the security model for the same and a UI redesign for an internal data inventory system for a public safety provider.
Education
Pembroke College, University of Cambridge (2001 - 2005)
I did my degree in mathematics. I received Class I Honours in my BA and Merit in my Part III (equivalent to an MMath)
Writing
I write extensively on a variety of subjects, mostly on my blog, which has a fairly respectable base of readers. Notable technical blog posts include:
- Some Things That Might Help You Write Better Software is an almost complete list of the generic advice I would give to anyone who wants to get better at software development.
- How to quickly become effective when joining a new company is an article about learning by doing and how to use it to get started much faster when joining a company.
- The economics of software correctness is an article on why I think bad software happens and what we can and can't do to fix that.
Other interesting things I have written include:
- Voting by Example is a short ebook I wrote to explain the complexities of voting systems and why they're interesting and important.
- Stargate Physics 101 is a work of fan-fiction, but it is primarily a story software project management and QA. I've been told by someone that they send it to their junior developers to teach them about the importance of testing!
- In a similar vein, Programmer at Large is a web serial about a possible distant future of software, chronicling the life and times of a maintenance programmer on board a starship as they battle with centuries old bugs in the plumbing system.
Fun Facts
- I mostly write Python at the moment, but I've written in a wide variety of languages. I learned to program in Standard ML and have been employed to write Java, Scala, C#, Ruby and C++. I can also write fairly passable Haskell and C, and have at least casual experience with many more languages.
- For some time I was an external maintainer for the Scala compiler and standard library. I primarily did bug fixing, but worked on a fair bit of code clean up and contributed to the design and implementation of the collections API.