You might have noticed that Hammer Principle has been down for a while. It should be back up now, and I’ll continue to try to keep it so.
The TLDR of what was going on is that its codebase is really old and hasn’t seen much love in a while. In particular all its dependencies hadn’t been updated in about 3 years. Oops. My bad. (It wasn’t running anything with security critical vulnerabilities that I know of – certainly none of the big news remote code execution ones that have come out in the ruby world in the last year or so are present). My last experience of trying to bring a similar set of dependencies in another project up to date was somewhere between “unpleasant” and “intractable”, so I’d been putting this off.
Which was fine, but about a month ago it started regularly eating up 100% CPU and making my hosting provider sad and fretful at me (sorry, dude), so I turned it off until I could look at it. I then repeatedly failed to getting around to looking at it.
I have now finally got around to looking at it after someone poked me about it recently. I decided the first thing to do was just to get the dependencies upgraded – running an ancient version of datamapper on an ancient version of ruby was basically never going to go well. To my embarrassment this turned out to be one of those tasks you put off for ages and then it turns out to be really easy. Took me a couple of hours last evening. I’ve also set up website monitoring and set things up so that unicorn workers will regularly be killed off rather than letting them get too big.
Then on further investigation I discovered that the varnish setup that was supposed to protect it from large numbers of visits was completely broken. It was completely broken in no less than three ways:
- The app was setting session cookies for everyone who arrived. This is something I’d fixed in the past. I’m not sure when it regressed
- Varnish was set up to run behind another web server, which was just silly
- The config pointing it to varnish was commented out and wrong
Err. Yeah. Ops: I’m still not especially good at it, but I was especially bad at it when I first set hammer principle up.
That should be fixed now.
Anyway, so hopefully hammer principle should be pretty stable now. If it isn’t, I’ll fix it. I’d like to do some more work on it, but I’ve got to admit that my track record of doing so is not astoundingly good, so no promises for now.