I awoke to the usual morning startup screen – diagnostics for my sleep, news, patches to my software, etc. Nothing very exciting. I was being gently chided for not having had a proper dinner yesterday – two protein bars after the gym really don’t count – but the system had replaced the worst of it while I slept and informed me I should just have a large breakfast to compensate for the rest. Other than that, all systems looked good.
I acknowledged the instruction for breakfast and spent another hundred or so seconds reviewing the data, but it was all pretty boring.
The correct thing to do at that point would of course have been to shower, get dressed, and go get some food as instructed, but something was nagging at me from yesterday’s work… I checked, and it was fine for me to put off breakfast for another 5 ksec or so (though I got the irrational sense that the system was judging me for it), so I decided to just go straight to work.
“Ide, how many temporary patches do we have in the system?”
“Four million, five hundred thousand, one hundred and seven.”
I squeaked slightly.
“Four million, five hundred thousand, one hundred and seven.”
OK, fine. I was expecting a large number. I wasn’t expecting it to be that large – I’d have thought maybe ten thousand at the upper limit – but it didn’t actually make a difference. Either way it was too large to handle manually, so it was just a question of getting the software right.
Still, more than four million? I know there’s nothing so permanent as a temporary fix, but that’s just ridiculous.
“Ide, what sort of time frame does that span?”
“The oldest is approximately 0.26 teraseconds old.”
That didn’t make any sense. The current patch system isn’t even that old. The trade fleet is barely that old.
“Ide, how are you defining temporary patch?”
“I have a complex heuristic subprogram that indexes logical patches from either the fleet system or imported software strata and looks at metadata and comments to flag them as temporary.”
“Oh, right. If you just look at trade fleet authored patches in the modern system which have been explicitly flagged as temporary how many are there?”
“One million, sixty two thousand and eight.”
“Ugh. And when is the oldest of those?”
“Approximately 0.15 teraseconds old.”
This was not going to be a productive line of inquiry, but curiousity got the better of me once again.
“OK, show me the oldest.”
ID: 3b2ca7b197f9c65e883ef177178e20e6bb14b... Author: Demeter [bim-bof 3 of the Entropy's Plaything née Wild Witless Bird] Revert-On: Closure of f265957e0a2... Add a flag that deliberately ties the theorem prover's hands by restricting the set of valid interleavings when running in time travel mode. Why? Well it turns out that *some* Linux descendants have a very peculiar idea of how x86 is supposed to work. This idea is backed up by neither the spec, nor by the actual physical machines that existed back when x86 was still a real thing rather than an interop standard. How did that happen? Well this wasted code comes from a descendant of the "Corewards Bound", who at some point introduced a bug in their implementation which made things run faster and didn't obviously break any of their software. When they found this problem a few hundred gigaseconds later they decided to patch Linux instead of their misguided grounder-inspired broken emulation software. Nobody backed it out later, it got passed down through three generations of ships, and finally got handed over to us and now we're stuck with it. This patch is stupid and should go away once the referenced issue is resolved.
I looked at the patch. It was some really hairy logic right down at the formalism layer for one of our emulators. I had absolutely no idea whatsoever what was going on in it. I didn’t know the language it was written in, I can barely read either x86 or the intermediate representation it was being compiled to, even with Ide’s assist, and besides I don’t know half of the maths required to understand why it was doing what it was doing.
The referenced issue was about patching Linux to not depend on this broken behaviour. It had reached almost a million words of discussion before just trailing off – I think given the timescales involved everybody who cared about it had died of old age. Or maybe we just lost touch with them – neither this code nor the code it patched were from anywhere in our direct lineage.
“Ide, how many services are running with this flag set?”
I breathed a sigh of relief. That was a much better answer than I was afraid of.
“When was the last time any of them had a bug?”
“No bugs have ever been reported in these services.”
“OK. How about in their service categories?”
“Approximately 80 gigaseconds ago.”
“What was it?”
“The Untranslatable Word passed through an area with an unusually high helium mix in the local interstellar medium. This increased the crash rate of the process by 15%, which triggered a maintenance alarm.”
“How was it fixed?”
“The alarm threshold was raised by 50%.”
OK. So I’d found a weird hack in the implementation of some extremely reliable services. My duty was clear: Do nothing, touch nothing, make like a diasporan and leave at extreme velocity. I was more likely to break something by trying to “fix” it than do any good by touching this.
Time to back up and look at the actual problem.
“OK. How many temporary patches are there that were created on the Eschaton Arbitrage which apply to some but not all running processes in a category, have a trigger to revert on a hardware change, but predate our last planetary stop?”
OK. Now we were getting somewhere.
I spent the next few ksecs triaging those nine manually. They all looked pretty harmless, but I bet there were some gremlins that they’d flush out when the relevant teams looked into them. That definitely wasn’t going to be my job though.
After that, I wrote up a wiki entry about this problem and filed an issue making some general suggestions for how we could improve the systems around this to stop it happening again. I wasn’t very optimistic it would go anywhere, but it was at least worth making the effort.
At which point, I decided it really was time for breakfast, and headed to the showers to get ready for the day.
I showered quickly, dressed, and spent a few hundred seconds dealing with my hair. I found the style my schema gave me a bit boring so I spent a little while tweaking the parameters to give something less symmetrical.
Eventually I got my hair decisions resolved, and headed for the common area for my much delayed breakfast.
Next chapter will be in two weeks (April 7th).
Like this? Why not support me on Patreon! It’s good for the soul, and will result in you seeing chapters as they’re written (which is currently not very much earlier than they’re published, but hopefully I’ll get back on track soon).