A case study in bad error messages

Consider the Python.

I’ve been writing a lot of it recently. It’s mostly quite nice, but there are some quirks I rather dislike.

You may have noticed, but I have strong opinions on error reporting.

Python doesn’t do so well on this front. This is sad given that it really loves its exceptions.

An example:

>>> float([])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: float() argument must be a string or a number

This error message has the lovely property of being both completely unhelpful and a lie.

It is unhelpful because it does not give you any information at all (not even a type) about the value you tried to convert to a float.

It is a lie because in fact all manner of things can be converted to floats:

>>> class Foo(object):
...     def __float__(self):
...             return 42.0
>>> Foo()
<__main__.Foo object at 0x27ac910>
>>> float(Foo())

I wonder how we could better design this message to mislead? I’m drawing a blank here.

This entry was posted in programming on by .

8 thoughts on “A case study in bad error messages

  1. Dave Hall

    A little harsh. For most users, who never bother with magic methods, its pretty safe to say that a string or a number are the only acceptable types. And knowing the line number, a quick print type(foo) will tell you the type you passed in.

    It could be a better error message, but this sort of thing rarely wastes my time in practice.

    1. david Post author

      A quick print type(foo) is not a valid solution. Errors happen in production, where you don’t really want to be inserting random print messages, and besides which you need to be able to debug them in production after they’ve already happened.

      The case where I’ve encountered this happened to be relatively easy to reproduce – it didn’t take me more than about 15 minutes of extra debugging – but in general I have wasted a lot of time debugging due to misleading or lazy error messages. Any individual one doesn’t matter much, but the aggregate effect of them is huge and I have very little sympathy for the claim that they don’t matter most of the time. Individual instances of it usually don’t matter most of the time. The aggregate effect of systematic behaviour which produces this sort of error message absolutely does.

  2. Veky

    I must say I don’t quite understand you. Do you say it would be helpful to know the _type_ of the thing you called float on? Well, there in front of your nose is the thing itself. I’m sure you’ll agree it’s a superset of information you need. If you’re saying there could be a variable, and you put different types of things in that variable at different moments, that’s very weird for a type-loving person. :-)

    About the “lie”: in a fantastically dynamic language where there are hooks for customizing absolutely everything, accessible from inside the language itself, every piece of documentation is a lie. Recently I had the pleasure of reviewing some code (put in a variable, say x) that could make _any_ expression containing x True. Yes, even “x is __builtins__ and x is lambda: 55”. Now you know that, try to document the “is” operator without lying. :-P

    1. david Post author

      The value is *not* right in front of me. What is right in front of me is an error message in a log somewhere. It does not contain the value.

      The correct error message is quite simple: “Cannot coerce value %s to float” % repr(value). Their mistake is in including “helpful” extraneous information that happened not to be true.

  3. Veky

    Do you, or do you not, have the _line_ in the source that causes the exception to be thrown? There you have at least the variable name that contained the value at that moment. If type is really what you’re interested in, it shouldn’t be hard to figure out, at least if you care about types enough to not put different types of things in the same variable.

    (If you want to really know the values themselves, you can: http://stackoverflow.com/questions/6061744/how-to-get-value-of-arguments-in-functions-at-stack. But not by default.)

    Your “correct” error message is not very helpful, if you’re a beginner and don’t know what a coercion is, and is still a kind of lie, since no coercion does have to take place. In your example above, in __float__ method, there is no connection at all between self and 42.0. You don’t even _use_ self in determining the return value. In my dictionary, that’s not coercion.

    The real error message would probably be “hasattr(type([]),”__float__”).__bool__.__call__() returned False”, but it just shows how ridiculous your wish is. Besides, I forgot about some other hooks (__getattr__, __getattribute__,…) that could be used to make even that error message a lie.

    1. david Post author

      Do you, or do you not, have the _line_ in the source that causes the exception to be thrown? There you have at least the variable name that contained the value at that moment. If type is really what you’re interested in, it shouldn’t be hard to figure out, at least if you care about types enough to not put different types of things in the same variable.

      This is a nonsense claim. Of course I put different types of things in the same variable. Polymorphism is a thing programmers actually use, and python is an especially dynamic language. Moreover, bugs happen and sometimes bad values propagate through the code as a result of that (in the actual example where I encountered this a None had unexpectedly leaked in because of a bad API elsewhere that returned None instead of throwing an exception).

      Frankly I can’t be bothered to argue this: I’ve told you what a sensible error message would look like here. It is obviously easy to implement. Yet you are repeatedly insisting that I’m a bad person for wanting this because I should be able to deduce it from the code. My point is that given sensible error message design I wouldn’t have to deduce it from the code. Sure in the toy examples I used to demonstrate the error message it’s easy to figure out, but “I shouldn’t have to design sensible error messages because programmers are perfectly capable of employing deductive reasoning and modifying their code to report additional information” is the very definition of bad error message design.

  4. Veky

    I don’t think you’re a bad person. I just think you don’t really understand how dynamic Python is. Yes, that concrete error message would have helped you in that concrete situation. But it’s still a lie, and it’s not helpful generally (for example, try `f=open(r”C:\some\path\to\file.txt”); a=[float(f)]`. I’m sure you’ll agree your error message is very confusing in that situation. And contrary to yours, this is the _real_ example: I’ve seen Python beginners try the above when wanting to read a list of floats from file).

    Face it, there really is no way to accomplish in a general and consistent way what you seem to want. When we encounter an exception, the only thing we can be sure of that we have encountered an exception. How it got there, what exact circumstances made it appear, and what portions of memory affected it, is just a guess. More or less accurate in various situations, but still a guess. (Even the whole traceback is just a snapshot, and we probably want the whole movie.) And experienced programmers know that. Try to answer: what was your first thought when you saw that error message (besides “Oh my, that’s a bad error message”:)? You probably didn’t think “Oh right, silly me. What made me think I could customize float by defining __float__ method? Of course float can be called only on strings and numbers”. Right? You knew that wasn’t the problem. The error message wasn’t for you. It was for the unfortunate kid typing the code above, trying to take float of a file.

    (Maybe you think you’re more important than that kid, and it is your right to think so. However, Python was designed specifically to be friendly to beginners — not to Python, but to programming itself. You just chose a bad language.:)

  5. Pingback: Best of drmaciver.com | David R. MacIver

Comments are closed.