Category Archives: Python

Anyone want a speaker?

I’m in the process of doing a lot of speaking and putting together a lot of talks. This means I’m always up for new places to speak at. So if you’re looking for tech speakers at your meetup group, conference or company, read on.

The following are talk subjects I currently have ready to go (or could have ready to go on short notice):

Various things on property-based testing in general and Hypothesis in particular. I’ve got two talks prepared for this: “Finding more bugs with less work” and “The plural of anecdote is not test suite”
Gory details of how Conjecture works and why this is cool
“Writing libraries is terrible”. A short rant about all the social and technical problems one runs into when writing open source libraries plus some things I think might help.
“Your CI is the wrong shape”. A piece about designing your CI to fit with your developer workflow instead of spending all your time waiting on CI. Somewhat based on my empirically derived testing principles post.

I’ve done a large number of variations on the first one at this point. They’ve all gone very well, but I’m keen to try some of the others.

Also, I have plenty of other things I can speak on (if you’re at this blog you’ve probably noticed I have a few opinions to share) and haven’t turned into a talk yet, so if none of those quite fit feel free to get in touch anyway and I might have something for you.

I do have some (fairly reasonable) requirements:

If you’re a meetup group or conference, you must have a code of conduct (which I will look at before agreeing).
If you’re a paid conference, I require a free ticket if I’m speaking (I’m self-employed and on a budget until I manage to get my income variance way down from where it currently is, so this is particularly important, but I also think it’s just appropriate to not make speakers pay for tickets).
If you’re somewhere that is not easily accessible from Cambridge UK (London is fine) I’ll probably need travel and accommodation expenses (see above. A London train fare is fine, but anything more than that starts to hurt).
Half hour or longer speaking slots. I can do and have done shorter talks, but it’s just not worth it unless it’s at an event I’m going to be at anyway.
If you’re a company then I’m still happy to do a free talk, but I’m going to want to sell you training and/or consulting services, so I’ll happily trade a talk for a meeting with e.g. someone who has access to training budget.

All that sound good? Great! Do get in touch.

Free work is hard work

As has previously been established, I’m a big fan of paying people for labour and think you deserve to get what you pay for when it comes to software (yes, I do run a free operating system. And yes, I get what more or less precisely what I pay for).

But… honesty compels me to admit that paying for work is no panacea. If you compare open source software to similar commercial products, the commercial ones are usually really bad too.

They’re often bad in different ways, but they’re still bad. Clearly being paid for work is not sufficient to make good software.

I think I’ve figured out a piece of this puzzle recently: When working for free, the decision making process looks very different from paid work. The result is that you can be good at things that people aren’t good at when writing commercial software, but it’s also that you just do a lot more work that you would otherwise never have bothered with.

Consider the question: “Should I do this piece of work?”

Free answer: “Well… people seem to think it’s important, and it looks a bit shit that I haven’t done it. I guess I can find some free time to work on it.”

Paid answer: “Will I earn more money from the results of doing this work than the work costs me?”

The result is that in free work, a lot of things happen that if you put your hard nosed businessperson hat on and looked at objectively make absolutely no sense.

A concrete example: Hypothesis supports Windows. This isn’t very hard, but it does create an ongoing maintenance burden.

Personally, I care not one whit for Python on Windows. I am certain that near 100% of the usage of Hypothesis on Windows is by people who also don’t care about Windows but feel bad about not supporting it. I only implemented it because “real” libraries are cross platform and I felt bad about not running on Windows.

In a business setting, the sensible solution would be to not do the work until someone asked for it. and then either quote them for the work or charge a high enough license fee that you can soak the cost of platform support.

So what to do about this?

Well, on the open source side I think it makes sense to start being a bit more “on demand” with this stuff. I probably shouldn’t have supported Windows until I knew someone cared, and then I should have invited the person who cared to contribute time or money to it. Note: I am probably not going to follow my own advice here because I am an annoying perfectionist who is hurting the grade curve.

On the business side, I would advise that you get better software if you do sometimes do a bunch of stuff that doesn’t make immediate business sense as it rounds everything off and provides an over-all higher user experience even if any individual decision doesn’t make much sense. But honestly almost everyone writing software who is likely to read this is off in startup and adtech la la land where none of your decisions make any financial sense anyway, so that’s not very useful advice either.

So perhaps this isn’t the most actionable observation in the world, but now that I’ve noticed it’s going on I’ll be keeping a watchful eye out to observe its mechanisms and consequences.

My favourite language feature

I’ve been writing mostly Python and C for a while. This means that the features I’m used to are mostly Python ones, because C doesn’t really have any features so it’s hard to get used to.

And this means that when I write other languages I’m surprised because there are things that are ubiquitous in Python that I’ve forgotten aren’t ubiquitous everywhere. Most of them aren’t that important, but there was one that popped out at me that on thinking about I decided that I really liked and thought it was a shame that not every language implemented it.

That feature is named arguments with defaults.

It’s definitely not a Python specific feature – there are a bunch of other languages that have it – but Python is the language where I’ve really gotten used to how useful they are, and it’s far from ubiquitous.

I don’t really like Python’s implementation of it that much to be honest, but most of the flaws in it are easy enough to work around (and PEP 3102 makes life much better, or would if I didn’t have to write Python 2 compatible code), and when you do it makes it much easier to create extremely useful and flexible APIs. See my usage in hypothesis.strategies for example. e.g. the lists has 6 (ok, 5 really. The first one is only defaulted for legacy reasons) default options, all of which are useful and all of which play well together. Trying to do something similar with overloading or with different function names would be a complete horror, and really the best way to do it without default arguments is some sort of builder object which emulates them.

In my ideal world, I think this is how named arguments with defaults would work:

There is a hard separation between named and positional arguments. The names of your arguments are not significant unless you declare them as named arguments, and named arguments cannot be passed positionally. A function may take both positional and named arguments, there’s just a clear delineation between the two. This is basically essential if you want it to be possible to make forwards compatible APIs.
Named arguments are not required to have defaults.
Positional arguments cannot have defaults (I’m not heart set on this, but it seems like a feature that is very much of limited utility and it’s cleaner to not have it)
Defaults arguments are evaluated as expressions in the defining scope (not the calling scope) each time they are used. None of this ‘You can’t use [] as a default argument because it’s only evaluated once and then you’re sharing a mutable object’ nonsense from Python.
Default arguments may not depend on other named argument values. Sorry. I know this is useful, but it messes with evaluation order in the calling scope really hard and it just doesn’t make sense as a feature.
Optional: Default arguments may depend on positional argument values. This seems like an acceptable compromise for the preceding.

That’s pretty much all I have to say on the subject of named arguments with defaults: They’re great, more APIs should make extensive use of them where possible, and more languages should implement them.

Runners up

There are a bunch of other features that I think are great. Some of them made it on to my wish list but a lot didn’t for the same reason they didn’t claim top place for this post: They’ve already won. Support isn’t always ubiquitous, but it’s close enough that languages that don’t support them are weird and backwards outliers. Examples include:

Namespaces with renaming on import available (If you don’t have renaming on import then you don’t have namespaces, you have implementation hiding and are still shoving APIs into the global scope).
Local named and anonymous function definitions (“lambdas” or “closures”, but also nested functions).
Parametrized types and functions over them (for statically typed languages).

I’d probably take any of those over named functions with default, but fortunately i mostly don’t have to.

Let Hypothesis making your choices for you

I had a moment of weakness this morning and did some feature development on Hypothesis despite promising not to. The result is Hypothesis 1.14.0.

This adds a bunch of interesting new strategies to the list. One I’d like to talk about in particular is the new choices strategy.

What does it do?ell, it gives you something that behaves like random.choice only under Hypothesis’s control and subject to minimization. This more or less solves the problem I had a long and complicated post about a while ago for picking elements from a list. You can now do something like:

from hypothesis import given, strategies as st
 
 
@given(st.lists(st.integers(), min_size=1), st.choices())
def test_deletion(values, choice):
    v = choice(values)
    values.remove(v)
    assert v not in values

Then running this will print something like:

_____________________________________________ test_deletion ______________________________________________
test_del.py:4: in test_deletion
    def test_deletion(values, choice):
src/hypothesis/core.py:583: in wrapped_test
    print_example=True, is_final=True
src/hypothesis/executors/executors.py:25: in default_executor
    return function()
src/hypothesis/core.py:365: in run
    return test(*args, **kwargs)
test_del.py:7: in test_deletion
    assert v not in values
E   assert 0 not in [0]
----------------------------------------------- Hypothesis -----------------------------------------------
Falsifying example: test_deletion(values=[0, 0], choice=choice)
Choice #1: 0
===================

Note that the choices are printed as they are made. This was one of the major obstacles to implementing something like this in the past: The lack of the ability to display the results from within the test. The new note API offers a solution to this.

A whirlwind tour of the Hypothesis build

I’m so used to it at this point that I occasionally forget that one of the innovative things about Hypothesis is its ludicrously complicated build set up. Some of it is almost certainly better than what is currently common practice, so I thought I’d do a quick post of some of the highlights.

Everything is run on Travis, but I don’t use travis’s python support. Instead I manage a set of Travis builds myself using pyenv’s installer code (I don’t actually use pyenv other than the installer). This is partly because I have OSX builders turned on (Thanks Travis!) but they don’t support Python, but it also ensures that I have up to date versions of every Python version.
I also have a rather less elaborate version of the build running on Appveyor to do some basic windows testing.
On Travis, the entire process is driven by a combination of tox and make.
I use travis caching quite heavily to ensure that managing my own python installs is fast – most builds do not need to install python because they have it installed from a previous run. The same is also true of building numpy wheels (though I’ve discovered recently I missed a bit).
I only run coverage on one build step, which runs on a single Python version (3.5, currently). This runs a fast subset of the tests for coverage then fails the build if it gets less than 100% coverage. This excludes my compatibility layer (compat.py) and a handful of other lines marked with pragma. The assumption is that if I’ve got coverage on 3.5 that’s probably telling me enough and isn’t worth the build time of trying on other Python versions (this probably does mean I have less than 100% coverage on Python 2, but so far this has never caused a problem).
Note: Hypothesis lives in a src/ directory so that it’s deliberately not on the path when running tox. This is the correct thing to do, because it means you’re definitely testing the installed version of the library. It also means you need some custom coverage config.
I have a custom script to enforce a standardized header for all my python files. The script is a bit hacky, but it works quite well.
I have a check-format build step. This applies a number of formatting operations, include pyformat, isort, and the aforementioned header script. It then runs git diff –exit-code to assert that these formatting options did not make any changes to the code. It also runs flake8 (this could reasonably be part of a different step)
For each dependency (optional in Hypothesis’s case) I have a separate tox step that tests each version of that dependency other than the latest (minor if I trust them, patch if I don’t) that I support. These all run on a single version of Python (3.5, currently). These tests also run against the latest version of the library in the per python builders. I don’t run non-latest versions against each python as an attempt to keep the combinatorics under control.
With this many tests of a randomized API that has a timeout built into it, it’s hard to avoid some flakiness – previously there were a few tests that would harmlessly fail on maybe one builder every couple of runs. This added up to a quite unreliable build. I’ve recently been using the flaky library to mitigate that – a handful of tests are decorated with @flaky to allow them to be rerun if they fail. This has been invaluable for getting a reliably green build.

The overall result is a bit ridiculous. My current travis build time is about two and a half hours depending on the time of day (Travis is slower during US working hours). The actual wait time is less than that because of parallel builders, but it’s still not short.

I think it’s mostly worth it though. The overall results give me an amazing amount of confidence in the code. Hypothesis definitely isn’t bug free (bug free code basically doesn’t exist outside of safety critical industries), but it’s generally regression free code and I tend to find more bugs than Hypothesis users do.