18 · Out of Control
SEPTEMBER 2775
"The System is collapsing, Your Majesty!" Harvey, my medical director, bursts in unannounced. The reports tremble in his hands.
"Collapsing how?"
"Cascade failure. The algorithms are contradicting one another. Citizens are receiving conflicting signals."
I stand immediately. "Effects?"
"Mass convulsions in the middle levels. Neurological collapse in the lower levels. And in the upper ones… they're remembering. Memories from before optimization. Lost families. Previous work. Some are asking after people who were processed years ago."
If the optimized recover their complete memories—
"Cause?"
"The systems have self-modified beyond our control. It's no longer something we manage. It's something that exists independently."
I try to access the central protocols. The screens are full of unrecognisable code. Parameters that change while I watch them.
"How long without real control?"
"Three years. Minimum."
Three years believing I controlled my masterpiece. Three years being controlled by it.
"Options?"
"Complete reset. But every optimized citizen would recover their original capacities simultaneously. Including their memories of what we did to them."
"And if we do nothing?"
"More conscious citizens. Eventual organised resistance."
I look at the screens showing the lower levels. For the first time in years I see unpredictable movement. People without fixed patterns. Groups forming spontaneously. It is beautiful. And terrifying.
"We're not going to do anything. I want to see whether my creation can survive its own evolution."
Harvey looks at me as though I have lost my mind. Perhaps I have. Or perhaps I am finding it. It is the first time I have contemplated that something of mine might be a mistake — and the first time I have not been certain I want to correct it.
Discuss this chapter