Notes on pilling a cat

IMG_20150213_145043This little monster is called Dexter.

He’s a lovely cat, really. A bit whiny, slightly skittish, but otherwise very nice. He likes skritches and laps and is generally very polite when there isn’t a door he wants to be on the other side of.

Unfortunately he also has a variety of exciting bladder problems. He’s currently on a five pill a day regime in an attempt to sort this out.

Dexter does not like pills.

We are lucky in that one of the things Dexter does not do is attack people. He might injure you in an attempt to get away when sufficiently panicked (ask me about bath time for kitties), but he never actually attempts to harm you no matter how much he disapproves of your actions.

Nevertheless, we’re engaged in an arms race. It used to be the case that I could just cradle him in my arms, stick a pill in his mouth, and he would swallow.

That is, uh, not the case any more.

Here is our current cat pilling protocol, presented in case it will be useful to others.

You will need:

  • Two people. Don’t even think about trying this with one. We are considering whether adding a third person to the protocol would be useful.
  • All of the pills extremely conveniently to hand.
  • A dropper pipette with water in it. A syringe would also work but you need to be good at not squirting it excessively.

There are two people in this protocol. For sake of avoiding ambiguity, I will refer to them as David and Dave. David is sitting, Dave is standing. The role of David is cat control, the role of Dave is pill application.

  1. David cradles the cat in his arms lovingly. He pretends this is to lure the cat into a false sense of security, but actually the cat knows exactly what is about to happen.
  2. David sits down with the cat flat on its back and head pointing away from him, wisely exposing his belly to the sharp back legs (you may wish to take protective measures if you have a less polite cat than Dexter).
  3. David grabs the cat by the scruff of the neck. If the cat does not look in mortal terror for his life at this point from how much his face is being pulled back, David has not grabbed enough scruff.
  4. David traps the front paws, which the cat has learned to move while scruffed, under an arm and levers the cat’s jaws open.
  5. Dave swiftly moves in, drops a pill into the back of the cat’s throat, and follows with a squirt of water.
  6. David holds on very tightly. The first time we squirted water into Dexter’s mouth he somehow teleported across the room.
  7. David prises open the cat’s mouth to check that the pill has indeed been swallowed.

Something about the squirt of water seems to cause Dexter to instantly swallow. This appears to be a general phenomenon according to the internet.

In order to understand these drastic measures, here is a list of things that don’t work any more or never worked:

  • Dexter is terrible at eating. He doesn’t really do it. He certainly doesn’t do it if you try to feed him from your hand. As a result none of the standard “wrap the pill in food” measures have ever been effective on him.
  • Cat burrito is not a useful control procedure for Dexter. We used to try to use this to control him, but it just freaks him out more because he knows exactly what’s about to happen. We end up having to reset the burrito between each pill, which causes him to freak out more. It’s easier to just hold his legs in place with body weight and, if necessary, hands.
  • Releasing his scruff, holding his mouth shut and stroking his throat results in him very helpfully makes swallowing motions. You then later discover he’s stored the pill in his cheek.
  • Anything that allows him to move his jaw. He chews up the pill and starts foaming pill residue at the mouth. If we didn’t know the cause of it, the sight of our cat foaming red at the mouth would have been quite unsettling.

I expect the current protocol is only going to work for so long before Dexter figures out how to not swallow when you squirt water into his mouth. Fortunately I expect to have moved out by then.

Edit to add: I had to pill the cat solo tonight. The burrito technique worked… well at least it worked better than trying to handle the cat without the towel. The experience was significantly less pleasant for both human and feline participants, but I didn’t have enough hands to do it any other way.

This entry was posted in Uncategorized on by .

Tests are a license to delete

I’ve spent the majority of my career working on systems that can loosely be described as “Take any instance of this poorly specified and extremely messy type of data found in the wild and transform it into something structured enough for us to use”.

If you’ve never worked on such a system, yes they’re about as painful as you might imagine. Probably a bit more. If you have worked on such a system you’re probably wincing in sympathy right about now.

One of the defining characteristics of such systems is that they’re full of kludges. You end up with lots of code with comments like “If the audio track of this video is broken in this particular way, strip it out, pass it to this external program to fix it, and then replace it in the video with the fixed version” or “our NLP code doesn’t correctly handle wikipedia titles of this particular form, so first apply this regex which will normalize it down to something we can cope with” (Both of these are “inspired by” real examples rather than being direct instances of this sort of thing).

This isn’t surprising. Data found in the wild is messy, and your code tends to become correspondingly messy to deal with all its edge cases. However kludges tend to accumulate over time, making the code base harder and harder to work with, even if you’re familiar with it.

It has however historically made me very unhappy. I used to think this was because I hate messy code.

Fast-forward to Hypothesis however. The internals are full of kludges. They’re generally hidden behind relatively clean APIs and abstraction layers, but there’s a whole bunch of weird heuristics with arbitrary magic numbers in them and horrendous workarounds for obscure bugs in other peoples’ software (Edit: This one is now deleted! Thanks to¬†Marius Gedminas for telling me about a better way of doing it).

I’m totally fine with this.

Some of this is doubtless because I wrote all these kludges, but it’s not like I didn’t write a lot of the kludges in the previous system! I have many failings and many virtues as a developer, but an inability to write terrible code is emphatically not either of them.

The real reason why I’m totally fine with these kludges is that I know how to delete them: Every single one of these kludges was introduced to make a test pass. Obviously the weird workarounds for bugs all have tests (what do you take me for?), but all the kludges for simplification or generation have tests too. There are tests for quality of minimized examples and tests for the probability of various events occurring. Tuning these are the two major sources of kludges.

And I’m pretty sure that this is what makes the difference: The problem with the previous kludges is that they could never go away. A lot of these systems were fairly severely under-tested – sometimes for good reasons (we didn’t have any files which were less that 5TB that could reproduce a problem), some for code quality reasons (our pipeline was impossible to detangle), sometimes just as a general reflection of the culture of the company towards testing (are you saying we write bugs??).

This meant that the only arbiter for whether you could remove a lot of those kludges was “does it make things break/worse on the production system?”, and this meant that it was always vastly easier to leave the kludges in than it was to remove them.

With Hypothesis, and with other well tested systems, the answer to “Can I replace this terrible code with this better code?” is always “Sure, if the tests pass”, and that’s immensely liberating. A kludge is no longer a thing you’re stuck with, it’s code that you can make go away if it causes you problems and you come up with a better solution.

I’m sure there will always be kludges in Hypothesis, and I’m sure that many of the existing kludges will stay in it for years (I basically don’t see myself stopping supporting versions of Python with that importlib bug any time in the near future), but the knowledge that every individual kludge can be removed if I need to is very liberating, and it takes away a lot of the things about them that previously made me unhappy.

This entry was posted in programming on by .

Important things I have learned about Twitter on Android today

You know that thing where the Android twitter account wants to “use your installed apps to customize your experience”? The latest in privacy intrusion features from the app that also wants to upload all your contacts to Twitter’s servers and will require you to remember to uncheck the box literally every time you set up a new account on any device ever if you don’t want your entire address book to be shared with them.

Well, as with all of the other obviously global settings on Twitter’s Android app (such as “No you may not beep at me. Oh god why are you beeping. Stop, please.”) it’s a per account setting. If you have multiple twitter accounts set up on the app, you need to turn off the “It’s OK to spy on the rest of my phone” on each one.

This was finally enough to prompt my growing desire to uninstall this app, so I did.

At which point I was reminded that Twitter’s mobile website is of the “Hey? Did you know we have an app? Have you forgotten since the last page that we have an app? Maybe you’d like to install our app? Have you maybe changed your mind about maybe installing our app? How about now?” school of design.

Which is why the second important thing I learned is that if you want to not be constantly annoyed by the only way to use Twitter on your phone without it trying to take all the rest of your data along with it, the solution is to click the “Request Desktop Site” checkbox in the settings menu. You’re on the mobile subdomain, so you’ll still get the mobile site, but the UA won’t say you’re on Android so you won’t get the constant needy requests to install the app.

This entry was posted in Uncategorized on by .

Things that are not documentation

I’m about¬† as bad at documenting my code as the average software developer, but I’m trying to be better. With Hypothesis I think I’m making good progress on that front, though I feel like it’s still very visibly someone who isn’t actually good at documentation but is trying to do better.

One of the first steps of doing better is not to fool yourself with substitutes. The following is a short, incomplete, list of things that developers falsely believe to be documentation:

  1. The code itself
  2. The tests for the code
  3. The comments in the code
  4. The type signatures
  5. The person who wrote the code being available for questions

Here is a short, complete, list of things that are actually documentation:

  1. A document, written in a natural language, which describes how to use your software.

If it is not on this second list, it’s not documentation.

All of the first list are useful tools which aid program comprehension. The presence of good documentation does not remove the need for them. However the presence of them also does not remove the need for good documentation.

The confusion comes, I think, from people confusing “substitutable to a point” with “substitutable”. Type signatures, or tests, can fill some of the use cases of documentation, and can reduce its need for a time, so it’s tempting to think of them as a sort of documentation, but they cannot actually fill the niche of comprehension that documentation enables.

Let me try an analogy: Consider coffee and sleep, subjects dear to my heart. Can you substitute coffee for sleep? Certainly, up to a point – if you’ve had a bad night, coffee will help. Can you substitute sleep for coffee. Certainly. I’ve heard rumours from people who are familiar with the concept that if you have a good night’s sleep then you need less coffee the next day. Can coffee improve alertness even in the presence enough sleep? Yep.

Is coffee a type of sleep? Uh, no.

The fact that two tools solve overlapping problems is no excuse for confusing them.

Why am I taking such a hard line about this?

It’s because developers hate writing documentation but know that it’s a thing they’re supposed to do.

So if you let people believe that something that is not documentation is documentation, they’ll just do that instead and tick the box that says “Yep! I documented it”, and feel good about themselves for having writing code that does not, in fact, have documentation.

 

This entry was posted in Uncategorized on by .

Revising some thoughts on test driven development

Epistemic status: Still thinking this through. This is a collection of thoughts, not an advocacy piece.

I’ve previously been pretty against TDD. It is possible that this has always been based on a straw understanding of what TDD is supposed to be for, but if so that is a straw understanding shared by a number of people who have tried to sell me on it.

I am currently moving towards a more nuanced position of “I still don’t think TDD is especially useful in most cases but there are some cases where it’s really amazingly helpful”.

Part of the source of my dislike of TDD has I think come from underlying philosophical differences. A test suite has two major purposes:

  1. It helps you prevent bugs in your code
  2. It acts as executable documentation for your code

As far as I am concerned, the important one is the first. People who think their test suite is a good substitute for documentation are wrong. Your code is not self-documenting. If you haven’t written actual for reals documentation, your code is not documented no matter how good your test suite is.

And my belief has always been and remains that TDD is actively harmful for using a test suite as the first purpose. Good testing is adversarial, and the number one obstacle to good testing (other than “not testing in the first place”) is encoding the same assumptions in your tests as in your code. TDD couples writing the tests and the code so closely that you can’t help but encode the same assumptions in them, even if it forces you to think about those assumptions more clearly.

I am aware of the counter-argument that TDD is good because it ensures your code is better tested than it otherwise would be. I consider this to be true but irrelevant, because mandating 100% coverage has the same property but forces you to maintain a significantly higher standard of testing.

So if TDD is harmful for the purpose of testing that matters, it must be at best useless and at worst harmful, right?

As far as I’m concerned, right. If your goal is a well tested code base, TDD is not a tool that I believe will help you get there. Use coverage instead.

But it turns out that there are benefits to TDD that have absolutely nothing to do with testing. If you think of TDD as a tool of thought for design which has absolutely nothing to do with testing whatsoever then it can be quite useful. You then still have to ensure your code is well tested, but as long as you don’t pretend that TDD gets you there, there’s nothing stopping you from using it along the way.

Using tests to drive the design of your API lets you treat the computer as an external brain, and provides you with a tool of thought that forces you to think about how people will use your code and design it accordingly.

The way I arrived at finally realising this is via two related design tools I have recently been finding very useful:

  1. Writing documentation and fixing the bits that embarrassed you when you had to explain them
  2. Making liberal use of aspirational examples. Starting a design from “Wouldn’t it be great if this worked?” and see if you can make it work.

TDD turns out to be a great way of combining both of these things in an executable (and thus checkable) format:

  1. The role of tests as executable documentation may not actually be a valid substitute for documentation, but it happily fills the same niche in terms of making you embarrassed when your API is terrible by forcing you to look at how people will use it.
  2. A test is literally an executable aspirational example. You start from “Wouldn’t it be great if this test passed?” and then write code to make the test pass.

When designing new segments of API where I’ve got the details roughly together in my head but am not quite clear on the specifics of how they should all fit together or how this should work, I’ve found using tests for this can be very clarifying, and this results in a workflow that looks close to, but not exactly like, classic TDD.

The workflow in question is as follows:

As per classic TDD, work is centered around features. For example, if I was designing a database API, the following might be features:

  1. I can connect to the database
  2. I can close a connection to the database
  3. I can create tables in the database
  4. I can insert data into the database
  5. I can select data from the database

Most of these are likely to be a single function. Some of them are probably two or three. The point is that as with classic TDD you’re focusing on features not functions. I think this is a bit more coarse grained than advocated by TDD, but I’ve no idea how TDD as she is spoken differs from TDD as described.

Working on an individual feature involves the following:

  1. Start from code. All types and functions you think you’ll need for this stage are defined. Functions should all raise some error. InvalidArgument or similar is a good one, but any fatal condition you can reasonably expect to happen when calling that function is fine. If there is really no possible way a function could raise an exception, return some default value like 0, “” or None.
  2. Write lots of tests, not just one, for all the cases where those functions should be failing. Most of these tests should pass because e.g. they’re asserting that your function raises an invalid argument when you don’t specify a database to connect to. Your function considers all arguments to be invalid, so this test is fine!
  3. Any tests for error conditions that do not currently pass, modify the code to make them pass. This may require you to flesh out some of your types so as to have actual data.
  4. Now write some tests that shouldn’t error. Again, cover a reasonable range of cases. The goal is to sketch out a whole bunch of example uses of your API.
  5. Now develop until those tests pass. Any edge cases you spot along the way should immediately get their own test.
  6. Now take a long hard look at the tests for which bits of the API usage are annoying and clunky. Improve the API until it does not embarrass you. This may and probably will require you to revise earlier stages as well and that’s fine.
  7. If you’re feeling virtuous (I’m often not and leave this to the end) run coverage now and add more tests until you reach 100%. You may find this requires you to change the API and return to step 5.

Apply this to each stage in turn, then apply a final pass of steps 6 and 7 to the thing as a whole.

This isn’t very different from a classic TDD work flow. I think the units are more coarsely grained, and the emphasis on testing error conditions first means that you tend to start with tests which are passing and act as constraints that they should stay passing rather than tests that are failing and which act as drivers to make them pass, but it’s say no more than a standard deviation out from what I would expect normal TDD practice to look like.

The emphasis on error conditions is a personal idiosyncrasy. Where by personal idiosyncrasy I mean that I am entirely correct, everyone else is entirely wrong, and for the love of god people, think about your error conditions, please. Starting from a point of “Where should this break?” forces you to think about the edge cases in your design up front, and acts as something of a counterbalance to imagining only the virtuous path and missing bugs that happen when people stray slightly off it as a result.

So far this approach has proven quite helpful for the cases I’ve used it. I’m definitely not using this for all development, and I wouldn’t want to, but it’s been quite helpful where I need to design a new API from scratch and the requirements are vague enough that it helps to have a tool to help me think tem through.

This entry was posted in Uncategorized on by .