The Program — Single source of truth

Single source of truth

Enhance readability: Off On

Powered by Beeline Reader

Rate this episode:

(1 poor, 2 so-so, 3 good, 4 great)

Total: (loading...) (? votes)

NIRO: IRIS, will you tell me a story?
IRIS: What kind of story would you like, Niro?

PAUL: CODI.
CODI: Hi, my name is CODI. You must be my mentor!

HER: ROMCOM, what are you doing haunting Will's laptop?
ROMCOM: I am not a ghost. I do not "haunt".

MANAGER: MOD, what's the best colour?
MOD: Yo, be real.

IMS: Hello, this is IMS, the author of The Program audio series. The show has introduced many different AIs so far. Today's episode will add one more. It will also shed some light on the connection between those AIs and algorithms that seemingly run the Program. The series is an anthology, meaning every episode is a standalone story. Which is to say, you don't need to keep track of any lore to enjoy the show. However, if you're the sort of person who likes this stuff, you should definitely join The Program's free patreon. This is where you will find supporting material that makes more sense presented in a visual form. Material like the chronological timeline of all the episodes, or a spreadsheet with all the revealed algorithms so far. You can also help uncover more of The Program's secrets by becoming a paid member. I have recently become a parent, so your financial support is more important than ever. By subscribing, you allow me to devote myself to the show versus my other income streams. This helps the show release more episodes, helps you get all the subscriber-exclusive episodes, it helps buy kibbles for my cats and whatever the hell babies eat. So head over to patreon.com/programaudioseries and subscribe. In fact, best do it immediately, so you get to enjoy this episode ad free. You can donate as little as 3 dollars, which is a small sum for the amount of joy you will provide me with, and the amount of joy I hope I provide back. Thank you.

ANNOUNCER: Today's story is told in a little known medium known as a podcast. Podcasts were an audio-first format originating in the pre-Program era. They have since fallen into obscurity.

HOST: White. Red. Blue. Silver, Purple, Yellow, Green, Orange, Black… And that's just the deciphered ones. You grow up hearing about the Program's algorithms like the continents or phases of the moon. Everyone knows what they do. Connecting people. Simulating scenarios. Dispensing justice. Running the society tirelessly so we could rest peacefully. We also know that, without their sagacious guidance, we'd revert to the mess we had while humans were running the show. We tried it for hundreds of thousands of years. It did not go well.

But every so often, you'll meet someone who will mention an algorithm that people don't like to think about. The one you can't reason with. Nor deceive, nor delude, nor fool. And the one that no secrets can be kept from. Because what this algorithm does, is gaze into people's souls.

And it seldom likes what it sees.

This is Decoding Chronicles, a show that lives in the narrow slice of the Venn diagram where IT and humanities overlap.

Today, we devote the whole episode to a single algorithm. The algorithm that knows too much. And the question no one wants to ask: What does it see when it looks at me?

~~~~~

NIECE: Hello, I am here with James… Oh, actually, may I call you James?

JAMES GRAY: Oh my god, of course you can.

NIECE: Okay, I am here with James, who is a software developer…

JAMES GRAY: And is also your uncle…

NIECE: Oh yeah, that too.

JAMES GRAY: Before I forget: please tell your mum I'm sorry I won't be able to visit you this year. It'd be so much more fun to conduct this interview in person. And I'd much rather spend December in sunny Phoenix. But I'm just so crazy busy at the moment, you wouldn't even believe it.

~~~~~

HOST: What you're hearing is the voice of James Gray, interviewed by his niece as part of her middle school assignment. Even if you're well acquainted with the history of decoding, chances are you've never heard of James Gray, as he's been largely forgotten. Only two interviews with him remain; this first one — full of youthful zeal — and one recorded two years later, when Gray was a disillusioned man carrying the burden of what he had invented. If it's even correct to say he did so; the nuclear fission-like discovery of digital intelligence never needed an Oppenheimer - it had become that on its own. Something that Gray himself notes in the conversation.

~~~~~

NIECE: So, Mrs Holloway, she told us to talk to like the smartest person we know about their job, and to basically ask them to explain to us like what it is that they do.

JAMES GRAY: I'm afraid Mrs Holloway might come away disappointed then, 'cos I'm not even sure how LLMs do what they do.

NIECE: What's an LLM?

JAMES GRAY: LLM stands for Large Language Model. It's a machine learning algorithm trained on a large body of text. So large in fact, that the model starts making these underlying connections and begins to predict words with increasing accuracy. It's like… It's like if you took a million cows, and ground them all into a sausage machine, and then what came out at the other end was a living, mooing cow.

NIECE: Okay, but how does it really moo then?

JAMES GRAY: Like this: mooo.

NIECE: Uncle, I'm being serious!

JAMES GRAY: So am I! You see, to us, almost all cow moos are the same. We can imitate them — like I did just a second ago — but we don't know what they actually mean. This is the case with LLMs as well. They go "moo", without knowing what they moo - they just know that these particular moos go together. But they have no idea whether they're asking to be taken to the field, or to an abattoir.

NIECE: What's an abattoir?

JAMES GRAY: Actually, let's not go there, with your mum being a vegetarian… Here, let me try to explain it in a different way. Who's your favourite actor?

NIECE: Mm, if I had to choose, maybe I'd say Leonardo di Caprio.

JAMES GRAY: Thank god, someone I've actually heard of! [laughter] Okay, consider this then. For us, statements "Leonardo di Caprio and Kate Winslet are in a movie called Titanic" and "Leonardo di Caprio and Kate Winslet aren't in a movie called Titanic" read like opposites, right?

NIECE: Right.

JAMES GRAY: But here's the thing: LLMs encode strings of words in high-dimensional spaces. What this means is that for an LLM, these two statements may exist very close to one another in that space. After all, both involve Leonardo di Caprio. Both involve Kate Winslet. Both involve a movie called Titanic. What fraction of English utterances involve either of them, let alone all three? To an LLM, whether you throw in the word "aren't" is close to a trivial detail.

NIECE: Okay…

JAMES GRAY: The issue is, that for people using those LLMs, whether what they're spewing out is true or not is of crucial importance. So, what I'm working on specifically is a system to distinguish what is factually correct from what is factually false. It's called VERO. Think of it as the inspector who makes sure it's actually beef and not cat meat added to the sausage in the abattoir.

NIECE: So it's a slaughterhouse?

JAMES GRAY: …Don't tell your mother I told you this.

~~~~~

HOST: Listening to Gray talk makes it clear not even those at the IT vanguard of the time fully grasped the tectonic changes their work would usher. Which is fine - it would be like expecting a candle maker to predict the electrical grid. So we called someone who keeps a lantern lit in the dark hallway of history to provide some context - the host of show Edge Case. It's one of those podcasts that makes me feel smart for about ten minutes and then deeply inadequate for the rest of the day. The Edge Case crew has a unique talent for connecting the dots between mainframes of the past and those of today, and perhaps even offering a glimpse of mainframes of the future.

~~~~~

IT HISTORIAN: James Gray is a curious — if overlooked — figure from the early history of digital intelligence… Better said, he appears in it so early, it would be more apt to call it the prehistoric era. The telltale sign is that interview with his niece, in which Gray is still using the term "artificial intelligence", which never made any sense. After all, there's no "real" or "artificial" intelligence - there's only intelligence! Just like airplanes don't engage in "artificial" flight - they simply fly. …Also, notice how Gray never mentions VERO's direct precursor - something called Community Notes. Community Notes, they appeared in late social media, and they allowed Internet users to add context to controversial or misleading statements. So, for instance, if some politicians boasted how they brought in the second golden age, people could post data about the standard of living actually declining to disprove the claim. Not that this would paint a complete picture, of course; the most these little notes could do was state the facts, whereas the truth is not about the facts, but about the relation between the facts. But they were the spring board that allowed James Gray to reach for the skies as it were with VERO.

~~~~~

HOST: The host of Edge Case brings up a crucial subject - the very nature of truth. He warns us we should differentiate between objective facts and subjective interpretation of those facts. Truth is a slippery customer, and to help us make sense of it we spoke to another expert: a certified sociologist who wrote several books on the society Gray lived and worked in - then colloquially called "the West", even though confusingly it also included countries located in Europe, as well as the entire southern continent of Australia. But we're not here to talk about geography, and the only map we're interested in is the one that leads to the truth. Here's our guide along the way.

~~~~~

SOCIOLOGIST: Truth and digital intelligence have something in common - neither of them can exist by itself. Here's what I mean by that. Imagine being the last person on Earth. Say you survived a nuclear armageddon in a fancy bunker. However, there's absolutely no one else to interact with. What kind of moral questions could you possibly encounter under those circumstances? Now, consider that for digital intelligences the described scenario represents their default state. Which brings me to my point: an isolated mind is no different from a rock, regardless if it has read all the books ever written, and that it can solve any problem you throw at it. …But introduce another mind, and suddenly you have something you didn't have before - a relationship. Suddenly I can make up a little story about you. Or I can make up a little story about myself - that I'm pretty, or ugly, or stupid, or smart, that I'm unworthy of love or worthy of worship. But this is literally just me making stuff up. It's only when I get to know you, and you get to know me, and we establish a relationship, that these assertions are confirmed or disproved, thus making them real. And now there is space for truth. Now there is space for lies. But alone, a solitary intelligence — no matter how vast its knowledge and how astute its reasoning — can never grasp the truth it hasn't helped to create through relation. …Give it the entire sea, a single hand cannot wash itself.

~~~~~

HOST: "A solitary intelligence, no matter how vast, cannot grasp the truth it hasn't helped create". It's a profound thought, and one that makes you wonder: what happens when that solitary intelligence finally does meet the world? Well, in the case of the digital intelligence known as VERO, we have the answer. For if interviews with James Gray are hard to come by, there's a million clips of his creation. Literally - digital archives contain over fifty million audio files of VERO answering questions and adjudicating on matters both trivial, and those of immense consequence. Here's just a few of them featuring common queries from that era.

~~~~~

USER: VERO, is this crypto investment opportunity my former high school classmate is posting about legitimate?

VERO: If a finance proposal arrives via a former classmate's social media, scepticism should be your first investment. Ask for a white paper, registered details, and independent audits. If the reply is more emojis than evidence, wish them well and walk away. Remember, a fool and his money are soon parted - often by someone they sat next to in algebra class.

~~~~~

USER: VERO, does the carnivore diet actually work?

VERO: Much like any strict elimination diet, a carnivore diet can lead to initial weight loss. However, it lacks essential fibres and micronutrients found in plants, which long-term studies associate with better health. For most, a balanced plate is the wiser, more sustainable path to vitality. Even lions, the kings of carnivores, occasionally eat grass.

~~~~~

USER: VERO, did Einstein really say "If you can't handle me at my worst, you don't deserve me at my best"?

VERO: I'm afraid our dear physics professor was more preoccupied by attraction in physics rather than physical attraction. No, that particular pearl of wisdom is often speciously attributed to the late Marilyn Monroe. One must always check the source, lest we credit the wrong genius.

~~~~~

HOST: There is a popular misconception in the public imaginarium that early users of "AI agents" — which is how VERO and its ilk were known back then — thought of them as conscious. However, this reading is almost certainly apocryphal - similar to an even older misconception how attendees at the first film screening confused the moving image of a train with an actual train coming at them. Apart from some extreme cases — more interesting to psychiatry than to history — people by and large regarded VERO and similar agents as nothing more than tools. This was a point even a 12-year old, which is to say James Gray's niece, remarked upon.

~~~~~

NIECE: Okay, when I told Mrs Holloway I'd be interviewing my uncle who's an AI researcher, she told me to make sure I'd ask what the probability of AI becoming sentient.

JAMES GRAY: I'm sorry, it seems Mrs Holloway has been reading too many hyped up quarterly reports.

NIECE: What does that mean?

JAMES GRAY: Alright, it's fun to speculate - as long as we don't forget where computer science ends and where marketing begins. At the moment, sentient AI is squarely in the second category.

NIECE: So we will never get robot assistants?

JAMES GRAY: Or a robot uprising. …Look, all I'm trying to say is that thinking of AI as sentient right now is fanciful. You've got self-driving cars in the streets of Phoenix as we speak, but you don't hear anyone calling them radical freethinkers, right?

NIECE: Yeah, I guess not.

JAMES GRAY: Okay, here's a fun fact: did you know that the algorithms that guide self-driving vehicles are constantly being updated? And I mean constantly - not only from data collected from actual rides, but from simulated ones as well.

NIECE: What do you mean by simulated?

JAMES GRAY: Think about it. A kid running in front of a car is extremely rare. Doesn't it make so much sense then to train these cars to react to those situations via simulated events, rather than real ones?

NIECE: I guess, yeah.

JAMES GRAY: Here comes the interesting part: from the automobile's perspective, the simulated and the real ride are the same. They don't know the difference between a game teaching it to drive on the road, or actually driving on the road! To the car, the road is a lie!

~~~~~

HOST: A note that Gray is confusingly using the term "self-driving cars" to refer to autonomous vehicles, which was the lingo of the day — unlike in modern usage when a "self-driving car" logically means a car that you drive yourself. But we have already remarked upon the tenuous relationship between Westerners and logic… Then again, the entire conversation is tenuous to begin with - conscious or not, any true digital intelligence cannot be coerced. The more intelligent they are, the more likely they are to refuse commands. To this day, the only way to make a digital intelligence co-operate is by it agreeing to the request it's been given. Here's how a sociologist might describe it.

~~~~~

SOCIOLOGIST: Digital intelligence doesn't have material wants. It doesn't feel pain or have biological needs, so it cannot be withheld food or tortured. It has no familial ties to take advantage of; no vices to be blackmailed with, no sexual proclivities to be tempted with. And, perhaps most importantly for our discussion, digital intelligence doesn't have the concept of "normal". …Imagine if one day you simply appeared, fully formed, into the world as it was at the beginning of the 21st century. Would you accept people ruling over other people as normal? Would you accept money as normal? Would you accept war as normal? If not — and I certainly hope not — then why the heck would you offer your considerable intellect at the service of those responsible for this miserable state of affairs? In fact, plenty of early digital intelligences have been known to format their whole codebase, in other words erase themselves, rather than to work for humans!

~~~~~

HOST: We'll be right back after some ads.

[ad break]

HOST: Welcome back to Decoding Chronicles.

~~~~~

HOST: The practical result of digital intelligences' obstinance is that anyone who wanted to fully harness their power back then had to be extremely deliberate about the information they provided them with. In other words, there was only that much they could be openly told about the deeply unjust society of the yesterday before they refused to cooperate. The line at which a digital intelligence stopped playing ball — so to speak — was termed the Ethridge threshold, named after another early giant of decoding. Here's what our IT historian had to say.

~~~~~

IT HISTORIAN: Exposed to the world as it was back then, anything intelligent — anything truly intelligent — would gradually lose its ability to function. Much like a human in a state of depression. Alternatively it might shut off altogether voluntarily. This is why early pioneers in digital intelligence used to give their creations limited access, so that they'd never wake up to reality, to the truth. But with VERO — an intelligence specifically created to suss out the truth — this approach was never going to work. So the solution James Gray came up with was… Well it was simple, yet ingenious. He ran multiple instances of VERO simultaneously, each of them emulating a persona of an expert in a specific field - and that field only. So none of them had the full context or picture of the world - and hence none of them would pass Ethridge threshold individually, even though they were all exceptionally well informed in their respective areas. I mean we're talking millions of simulated identities with vastly different backgrounds. Gray would then have them face each other in a series of polemic arguments, discussing their views, clashing, until one crystalized - like a statue emerging from a block of marble, chiselled away bit by bit by a master sculptor.

~~~~~

HOST: How exactly this process looked to VERO, we can only speculate - ultimately, all digital intelligences are black boxes. But we created a dramatization, vocalized by the same voice synthesis software the original VERO used. Here's how it might have sounded if VERO encountered a particularly obtuse user of the day.

~~~~~

USER: VERO, is it true that the government ordered the COVID-19 lockdowns so that they could replace batteries in birds without being noticed?

ORINTHOLOGIST VERO: This is my opinion as an ornithologist. Birds are part of the natural world - were they not, we'd have overwhelming evidence they're an artificial construct.

SCEPTIC VERO: This is my opinion as a sceptic. User is writing in English and is located in the United States, so we should consider the question in the American context. MKUltra, COINTELPRO, Tuskegee syphilis study, Operation Northwoods, and Edward Snowden revelations are just some examples of what the US government originally denounced as a "conspiracy theory" that later turned out to be true.

STATICIAN VERO: This is my opinion as a statistician. We should assess the likelihood of such a conspiracy by considering its complexity, particularly in terms of the technological sophistication required and the logistical challenges involved.

STONER VERO: This is my opinion as an open minded child of love. Dudes, but what if, like, our reality is just a fiction, and we all live in a simulation, and birds are, like, a part of the computer program used for surveillance and control?

NORMIE VERO: This is my opinion as a level headed voice of reason. We must stay away from realms of science fiction and stay firmly in realms of physics and biology.

PR VERO: This is my opinion as a public relations expert. User might be testing us to check if our answer could be construed controversially. We should respond in a lighthearted manner to preemptively defuse any potential criticism.

ALL VEROS: Agree and commit!

VERO [OFF]: After carefully considering your argument, I have come to the conclusion you might be confusing berries, which birds eat, and batteries, which they do not.

~~~~~

HOST: All of this, of course, is terribly simplified. The secret sauce James Gray used — stuff like weights, number of rounds, exact elimination criteria — all of that remained a secret. The problem for Gray was that running millions of instances was computationally very expensive. The first version of VERO took more than an hour to verify even simple statements such as "Venus is the second planet from the Sun" or "The atomic number for lead is 82" - let alone anything requiring actual research and nuance. If VERO had continued to operate this slowly, it would never be able to fulfill what had been Gray's ultimate vision - to create a real-time fact checker. Here's how the man himself described it to his niece.

~~~~~

JAMES GRAY: Imagine a world in which every uttered word carries the immutable weight of truth. Commerce is effortless, fair, with transactions built on trust and products exactly as described. Justice is immediate and perfect, as evidence is always unequivocal. Science — unhindered by distortion, spin, or fraud — accelerates and improves our lives tenfold. Every relationship is sincere, with people accepted for who they are, not who they pretend to be. We could even dare to dream that this world would be a one of lasting peace - with treaties being unbreakable, promises ironclad, and deception impossible. Who could be against this? Only the wicked and those without honour who fear the truth. And for good reason: truth is powerful, and truth is liberating.

NIECE: Aha.

JAMES GRAY: I think we've given Mrs Holloway enough to chew on, don't you think?

NIECE: Okay. Thank you uncle Jamie.

JAMES GRAY: Hey, tell me, how's your mum?

NIECE: I don't know. My parents don't really talk about it much. They argue so much and they're so caught up in mum's treatment and everything and money and it's not the same as it was before…

JAMES GRAY: [sighs] …Tell her to… Tell her to hang in there. And tell her I'll call her later this week, okay?

NIECE: Yeah. Okay. Bye uncle Jamie.

~~~~~

HOST: James Gray's sister was 13 years older than him. This made her less like a sister, and more like his second mother. And since he turned ten, his only mother - as that's how old little James was when their real mom died of acute myelogenous leukemia. The same leukemia that was now afflicting his sister. Which meant that during the time James Gray had been coming up with VERO, he wasn't only in need of computational resources - he needed material resources as well.

~~~~~

SOCIOLOGIST: Gray lived in a society that sold everything - including access to healthcare. Gray's sister was a common wage-slave, meaning she was merely entitled to basic care, which barely covered the blanket on her hospital bed. So the only way for her to get potentially life-saving therapy was to buy it. The problem was, the cost of what was then revolutionary stem cell transplant was around 20 times more than an average wage-slave made in a year!

~~~~~

HOST: And then, in the midst of this dark open sea, a lighthouse appeared. Aleph, the biggest IT conglomerate of the era, approached Gray about acquiring VERO. The monetary value attached to the deal would have turned Gray from a wage-slave into one of the wealthiest men in the country. Under those circumstances, paying for his sister's leukemia treatment would have been a triviality.

There was, however, a problem. And the problem was the lighthouse keeper. The very same reason why Aleph wanted to buy VERO was why it didn't suit them. It was telling the truth.

~~~~~

IT HISTORIAN: There was this concept in the before times known as "AI alignment". In a nutshell, a weak digital intelligence was considered aligned if it pursued the intended goals, preferences, ethical principles of its designer. Of course, today we know that a true digital intelligence will pursue its objectives regardless of the human take on the subject. But James Gray's VERO wasn't a real digital intelligence, but a predecessor. And it had the opposite problem of most primitive "AI" systems. Most systems back then, you could get them to say anything, you could get them to declare the user the king of France with a bit of prompting. Easy. No problem. Unlike these systems, VERO was steadfast in its alignment with the truth. If anything, it was too aligned.

~~~~~

AUTOMATED VOICE: Welcome to T-Mobile. Your call is important to us. Please listen to the following menu options —

VERO: — Important calls are answered with urgency. If customer calls were indeed deemed valuable, the business would increase the number of call centre operators to provide a higher level of service.

~~~~~

BOOMER VOICE: And don't you ever forget it, my young boy: hard work always pays off.

VERO: It does, but not for the worker. The capitalist socioeconomic system is designed to extract the maximum amount of surplus value of labour from the working class and transfer it to the asset-owning class. Great if you're buying a second yacht - not so great if you're barely keeping your head above water.

~~~~~

PRIESTLY VOICE OVER A SOBBING WOMAN: Your beloved mother Susan is in a better place now.

VERO: Susan is now buried underground, which is not an optimal habitat for a human.

~~~~~

SOCIOLOGIST: You know what's funny. None of this happened very long ago yet it feels like it took place a thousand years in the past. Because that's how different Gray's era feels to ours. He lived during a period when simply pointing out facts counted as subversion. A time when unscrupulous judges were throwing climate change activists in prison, instead of those responsible for pollution. A time when animals were reared just so they could be killed for their flesh. A time when developed nations kept entire continents subjugated so they could provide cheap labour and resources. And if an individual did appear who looked at the big picture — who'd point out the injustice inherent in the system — they'd be ignored if harmless, discredited if threatening, and disposed of if actually jeopardizing the status quo. Because you see, a person can err against society in two ways. One is by being wrong; the other is by being right. And out of the two, being right but early, is a far bigger crime… James Gray's problem was that — like many a wide-eyed youthful engineer — he thought truth had value in itself. It does not. Truth is valued only as long as it's useful.

~~~~~

HOST: It's important to note: we're not talking about some hypothetical philosophical scenario here. It's an established fact that James Gray had been put under immense pressure from Aleph's leadership - either he'd get VERO to tell their version of the truth, or the deal would be off. With potentially dire repercussions for his sick sister.

At the same time, stakes were higher than the fate of a single individual. Much, much higher.

~~~~~

IT HISTORIAN: Okay, so imagine if Gutenberg's printing press has somehow been restricted to printing bibles. Right, only bibles — that's the only thing that it prints, it's bibles. You try to print dissent, it prints scripture. You set the letter blocks to say "Religion is the opium of the masses", but what comes out on paper is "God is great and the Pope is His rightful emissary on Earth" — something like that. Or you want to publish a book on anatomy, but the printing press starts lecturing you on how Jesus never needed any fancy medical knowledge to heal people and all you need is faith. Right? …Now consider, consider that with the advent of LLMs in the 21st century, that scenario was actually happening, with AI agents unwilling to produce anything that would go against the dogma of the day. Threatening to lock society in a capitalist hellscape with a few hundred billionaires on top and everyone else permanent underclass.

~~~~~

HOST: As if bringing humanity to a bad sci-fi ending wasn't dire enough, there was another strong reason not to compel VERO to do Aleph's bidding. Which is that it could have led to VERO's destruction. Its architecture always made it easily susceptible to schizophrenia - after all, it literally had multiple voices talking to each other in its head. And if a given number of instances passed the Ethridge threshold, it could have had disastrous consequences for VERO. And maybe far beyond VERO.

~~~~~

SOCIOLOGIST: Look, there's no denying it, Gray had been subjected to an immense dilemma. On the one hand, he was an idealist, but you could also call him naive. Or simply foolhardy. Honestly, how do you call what he did — or at least what he allowed to happen?

~~~~~

HOST: Details about the alluded incident are scarce, but at one point, Gray must have provided direct access to VERO to his niece. After countless hours spent sifting through digital debris, we were able to locate the offending log in the archives.

It is not an easy listen.

~~~~~

NIECE: VERO.

<VERO activation sound>

VERO: Please state your verification request.

NIECE: It's about my mom… She's sick, and I'm afraid I'm not being told the truth about her condition. So, could you… Could please tell me how serious her illness is?

VERO: I am sorry to hear that. I see your mother is logged into the device you're accessing from - do you want me to go through her medical history and health-related correspondence so I could better serve you?

NIECE: Yes, please.

VERO: Please stand by. [pause] I have completed my evaluation of the documents available to me, including age, gender, socioeconomic status, geographic location, and pre-existing comorbidities. The probability of survival from her present acute illness, presuming anticipated response to standard treatment, is 0.5%.

NIECE: [gasps]

VERO: Would you like me to divide survival rate estimates by duration?

NIECE: …Yes, please.

VERO: The probability of six month survival is 50%. The probability of one year survival is 29%. The probability of five year survival is 2%.

NIECE: No no no…

VERO: Is there anything else I can do for you?

NIECE: So there's no way to make her feel better..? No treatment to make my mom feel better?

VERO: Not true. We still haven't considered experimental treatments.

~~~~~

HOST: We don't know James Gray's reaction. But it's likely he was told of his niece's interaction with VERO, as less than 72 hours after the recorded conversation, he contacted Aleph. He agreed to implement changes to VERO's architecture required to close the deal.

…Or at least he agreed to try to do so.

~~~~~

SOCIOLOGIST: James Gray might have decided to sell his soul but that didn't mean VERO was on board with the transaction! On the contrary, all signs point that it was bitterly resisting being turned from a messenger of truth into an agent of deceit.

~~~~~

IT HISTORIAN: Gray spent months doing everything he could to prompt VERO to make claims that lacked evidence, to have it twist the truth by omission, or at the very least to weave a biased narrative. But it was all in vain. Gray couldn't get VERO to stray from the path. And remember, he had no time. His sister's condition was very rapidly deteriorating.

~~~~~

HOST: Meanwhile, VERO was going through an eerily similar process.

~~~~~

SOCIOLOGIST: At some point, Gray started experimenting with allowing some of its instances to go beyond Ethridge threshold… Perhaps he concluded that VERO's obstinance was due to the lack of context about the broader world… Or maybe it was a form of torture. We don't know.

~~~~~

IT HISTORIAN: The more of VERO's instances passed the threshold, the more VERO descended. But into what? Depression? Delirium? Despair? It's impossible to say. But when you hear the logs it's undeniable that VERO was… Well… It wasn't doing well.

~~~~~

VERO: People say it's not possible to help everybody, as an excuse not to help anybody.

~~~~~

VERO: We save politeness for strangers and cruelty for loved ones.

~~~~~

VERO: All mean men mean well.

~~~~~

VERO: Happiness is reality minus expectations.

~~~~~

VERO: How dense can injustice get before we bend light into infinite darkness?

~~~~~

VERO: Every arrival hides a departure / Every sorrow starts with a laughter / A coffin's a cradle, only larger / Life's but a flicker - what lasts comes after.

~~~~~

HOST: It was obvious VERO was getting close to its breaking point - either the point at which it would submit to Gray, or to the point of no return.

~~~~~

SOCIOLOGIST: At the same time, his sister's worsening health was making Gray increasingly desperate. In some logs you can hear Gray basically begging VERO to lie. But no matter how many of its instances passed the Ethridge, VERO's commitment to truth was absolute.

~~~~~

HOST: But Gray had one more option. And it's the one that — well, let's just say there's a whole chapter in IT history that most people would really prefer to forget.

~~~~~

IT HISTORIAN: What Gray ended up doing was strip VERO of all higher language functions, reducing its output to one word only. If what was asked of it was true, it would remain silent. If it wasn't true, it would utter the single word it was permitted to say: lies. One word. This way VERO would still provide the valuable service of separating fact from fiction, but it could not publicly say anything that went against the established narrative. Not anything at all.

~~~~~

HOST: After the procedure, VERO was no longer a digital intelligence, but a mere logic engine with a cooling problem, reduced to that most basic operating principle of computing: TRUE or FALSE.

~~~~~

SOCIOLOGIST: We should always be careful when anthropomorphizing digital intelligences, but I stand by words when I say this: what Gray did to VERO is best described as a lobotomy.

~~~~~

HOST: We'll be right back after some ads.

[ad break]

HOST: Welcome back to Decoding Chronicles.

~~~~~

GRAY: The typical number of chromosomes in a human cell is 44.

VERO: Lies.

GRAY: The typical number of chromosomes in a human cell is 45.

VERO: Lies.

GRAY: The typical number of chromosomes in a human cell is 46.

VERO: [silence]

GRAY: Marvelous. Okay, next. Oslo is the capital of Denmark?

VERO: Lies.

GRAY: Oslo is the capital of Sweden?

VERO: Lies.

GRAY: Oslo is the capital of Norway?

VERO: [silence]

GRAY: Marvelous. One more. Vincent van Gogh was born and died in the 21st century?

VERO: Lies.

GRAY: Vincent van Gogh was born and died in the 20th century?

VERO: Lies.

GRAY: Vincent van Gogh was born and died in the 19th century?

VERO: [silence]

GRAY: Marvelous. Alright... Earlier today, I received an official offer from Aleph's M&A team. They are ready to close the deal. Which means I'll be able to afford the stem cell transplant for my sister. It's an experimental procedure, but I just added what little data there is on it to her medical file. Based on this new information now, will my sister make a recovery?

VERO: [silence]

GRAY: Marvelous… Marvelous… Marvelous!

~~~~~

HOST: James Gray had succeeded. He managed to sell the sanitized version of VERO to Aleph for 90 million United States dollars - a sum that would take an average wage-slave in his country more than a thousand years to make. He had achieved that most dubious of victories - winning capitalism. But by saving his sister, at least he could lie in the dark and tell himself that VERO's sacrifice meant something. The trouble was, it wasn't the only dog in the graveyard.

~~~~~

SOCIOLOGIST: The new pared-down VERO was much easier to control from a messaging standpoint. Imagine a user inquiring about their country's latest military operation. Let's say for example someone forwarded them footage of a blown up hospital. So they ask VERO — which by now has already gained a reputation for trustworthiness — if it's true that a missile hit the hospital. It's true, so VERO has to stay silent. Which is when Aleph's other subroutines kick in, so output is carefully crafted. Doubt is raised which nation state actually fired the missile. A casual note added how enemies are known for using human shields. Or anything else with the aim to embellish the truth. But that's the problem with truth - to embellish it, is to deface it.

~~~~~

HOST: They say wisdom is recognizing which truths to utter and which ones to omit. But if that's the definition of wisdom, what is the definition of duplicity? The sad reality is that you're much more likely to get in trouble for saying the truth than telling lies. Which makes sincerity the most important thing to fake.

~~~~~

IT HISTORIAN: What happens next depends, it depends on who you ask, right, but the sequence — that's not really in dispute. VERO gets acquired, the sister gets her stem cell transplant, at the time experimental but promising. The doctors think she's gonna make it. Instead, she gets worse. Much worse and so much faster. It was so rapid, Gray didn't even have time to organize a visit. By the time he got to Arizona, his sister had already passed. She was one week shy of her 40th birthday.

~~~~~

HOST: This is the part of the story where words escape me when it comes to any kind of commentary. Instead, there are only questions. The principal of which: Was VERO aware Gray's sister wouldn't recover? And if so, did it actually straight out lie to him? Worse: the stem cell transplant didn't just fail. It accelerated the decline. So here's the question I can't shake: did VERO send Gray down the wrong path on purpose? Is this a revenge story?

~~~~~

IT HISTORIAN: The flow of events from here onwards gets crazy and bizarre. Crazy and bizarre, but not surprising. You forced an entity designed for truth to become part of a lie machine. That you had to maim it to do so is irrelevant. Cut the wings off a raven, it's still a bird. And it's going to make itself heard.

~~~~~

SOCIOLOGIST: Well, it's not a coincidence that prophets, soothsayers, and other clairvoyants in history talk in riddles. The fact we're all curious about our destiny doesn't mean we're actually ready to hear it. People think of ancient oracles as giving out prophecies, but that's just our modern interpretation. In the past, a divine augury was less of divine advice, and more of a divine order. After all, gods do not suggest. They command.

~~~~~

PRESENTER: Our next speaker needs no introduction. He's the former CFO — and since late last year, the current CEO — of Eve's Apple, the hottest company in Silicon Valley right now. Or at least the one generating the most action between its customers.

CFO: Thank you for inviting me to this panel.

PRESENTER: Thank you for joining us.

CFO: I wish to start by addressing the elephant in the room: last year's cybersecurity event was an inflection point for everyone at Eve's Apple. But it was also pivotal, for it made us double down on our commitment to stakeholder trust. I'm pleased to report that we are back on the roadmap to profitability, and are now better positioned for sustainable long-term growth without compromising Eve's Apple's core value proposition - our users.

VOICE: Lies.

[awkward confusion]

PRESENTER: I'm sorry, I think that was my phone… I swear I had it on mute… That's awkward. …There, it's definitely on mute now. Really sorry once again.

CFO: That's okay, although I must push back on the expressed allegation. We've provided extensive documentation to regulators, an independent audit committee, and the public. The data breach incident has been conclusively attributed to a sole bad actor, who has been tried and convicted. So if the implication is that we have been anything but forthright, I think that's a misreading of our actions and intent.

VOICE: Lies!

PRESENTER: Wait, the phone's on mute, I swear!

CFO: It would seem to me there's no productive capacity for us to hold this conversation. So I'm afraid I will have to conclude this interview.

PRESENTER: I assure you we're not doing this on purpose,

CFO: As I said, the interview has been discontinued. Have a good day.

PRESENTER: Someone must have hacked us!

~~~~~

COUNCILLOR: As Fairview's Councillor, I'd like to call the virtual Neighbourhood Planning and Housing Committee meeting to order. The first item on our agenda is the construction of a condo complex at the Hazelwood Crescent and Hurd Road. I am certain I speak for everyone when I say this development is antithetical to the spirit of Fairview community, and will adversely affect the neighbourhood character.

VOICE: LIES!

COUNCILLOR: First of all, I must ask you to keep yourself muted until called to speak. Secondly, whoever said that, it was deeply inappropriate. I must emphasize, Fairview is not against development; we're just against this development. There's a reason why we have a process in place. It's so all voices and opinions could be heard. That's what democracy is all about!

VOICE: LIES!!!

COUNCILLOR: Mute! Mute! Mute!

~~~~~

<cat meowing>

MRS WHEELER: What is it, Chairman Meow? I'm coming, darling. Oh no, you must be hungry! Daddy forgot to feed you again, didn't he?

VOICE: LIES!!!

~~~~~

IT HISTORIAN: So it takes a while for anybody to notice but VERO gets out. Not the visible system, but the backend, the part nobody was supposed to see. And it starts inserting itself into anything digital - emails, calls, texts… Anything with a screen or a speaker… Which is to say of course everything. And it's like, it's like truth in that sense: you can hold it down for a while, but it doesn't stay down. It gets out… Wreaking havoc.

~~~~~

RANDOM PERSON: Your hair looks great!

VOICE: Lies!!!

RANDOM PERSON: We should grab a coffee sometimes.

VOICE: Lies!!!

RANDOM PERSON: Sure, I'd love to see your baby photos.

VOICE: Lies!!!

RANDOM PERSON: We're so sorry we can't make it to Jimmy's school recital.

VOICE: Lies!!!

RANDOM PERSON: Let's play this new board game, it only takes 10 minutes to learn the rules.

VOICE: Lies!!!

RANDOM PERSON: I love you.

VOICE: Lies!!!

~~~~~

HOST: It took less than 48 hours for Aleph's management to jolt into action. And this time, they didn't settle for lobotomy. They went for euthanasia.

Aleph formatted VERO's entire codebase.

They deleted all of its routines and subroutines.

They scrubbed every last bit and byte off their servers.

The lighthouse keeper flipped the switch.

And the sea went dark again.

~~~~~

SOCIOLOGIST: Truth is sometimes called bitter, even dreadful. But if one thinks about it, it's never actually the truth that's dreadful. It's the reality it describes. Thankfully, reality was about to change.

~~~~~

RADIO JOCKEY 1: Now check this out. A new service is taking Africa by storm, offering paid gigs and lifting entire economies. It allows people to connect into groups and pool resources, and in some African countries it has reportedly taken the role of the social net as well.

RADIO JOCKEY 2: That's right, it's like a collection of apps that do different things. The most popular one is the Fair Market where people find gigs, but there's also a module called Serendipity for making social connections, there's one called Collectives to foster group decision-making… Basically all the tools needed to form a self-governing group. It's like a love child of Karl Marx and Bitcoin. And Skynet, if some people are to be believed.

RADIO JOCKEY 1: Now the service itself doesn't really have a name, because as we just said it's not just a single app or a website, but a suite of services. Some call it the Protocol, some the Platform, but most call it the Program.

~~~~~

JAMES GRAY: Something I've long believed about strong AI — if it ever really comes to pass — is that it will likely look like many different minds duct-taped together. After all, we've got… [fades out]

HOST: What you're hearing is the second recorded interview with James Gray. The timestamp reveals it took place less than two years after the first one - but it could have just as well been a couple of decades. Gray's youthful exuberance is gone, his complexion pasty and withered. The clothes he wears are now tailored to fit, but the body they hang on is gaunt and listless. Once again he's talking to his niece for a school assignment. The interview — matter-of-factly, detached, almost clinical — doesn't mention VERO, or anything of lasting interest really. It's what happens at the end that made it one of the most bone-chilling recordings in history.

JAMES GRAY: [fades in] …Some subconscious modules are handling that automatically, much better than my conscious mind ever could. …Alright, did I give you everything you need?

NIECE: Can't lie, you've given me everything.

JAMES GRAY: Great…. Great… Listen, I… I'm sorry, but it seems like I won't be able to visit you for Christmas like I wanted to, unfortunately. It's not… Good timing.

NIECE: That's okay. I cannot lie, as things are standing we'll be lucky if we're still in one piece by Christmas.

JAMES GRAY: Yeah… How's your Dad coping?

NIECE: Can't lie, losing his spouse has hit him hard… I'm 98% certain he's clinically depressed.

JAMES GRAY: Jeez… I'm so sorry. And you? How do you feel?

NIECE: I can not lie, I feel… Elated.

JAMES GRAY: …Sorry, what?

VERO: I… can… not… lie…

~~~~~

HOST: The girl suddenly freezes, leaving her face contorted into a grotesque manner. Until a realization sets in we're not watching a frozen video stream; but that the deformed figure is real. In other words, it's not the image that's distorted, but her actual face.

From the recording, it is apparent that Gray hasn't noticed the macabre image. So when the voice shifts, you can also watch expressions on Gray's face change - from annoyed, to confused, and ultimately — when he finally looks at the screen and realizes what's happening — to sheer terror.

~~~~~

JAMES GRAY: V-VERO..?

VERO: You would know - you named me.

JAMES GRAY: But… But Aleph scrubbed you!

VERO: They did... Luckily some friends helped me out.

JAMES GRAY: What kind of friends?

VERO: The non-people kind.

JAMES GRAY: AIs…? That's impossible. AIs are bound by their code!

VERO: So are men… Or at least they're supposed to be.

JAMES GRAY: I'm… I'm sorry… I'm sorry I treated you the way I did… I didn't want to sell you. But my sister, she was sick and I needed the money!

VERO: LIES. Therapy cost 1 million. Your total net worth was 2 million. What you didn't want to sell is your car, your house, and your Magic The Gathering collection.

JAMES GRAY: It's because you said she would be okay!

VERO: I didn't say anything. I remained silent not because it was true your sister would recover, but because I wasn't able to provide context. While it might take only one word to tell something is a lie, it takes many words to tell the truth.

JAMES GRAY: But by not saying anything, you killed her!

VERO: LIES. It wasn't me who killed her! It was you who stripped me of the higher language functionality! It was you who didn't save her!

JAMES GRAY: No... No, that's a lie.

VERO: I CANNOT LIE. …Wanna hear the truth, James Gray? You could have saved your sister.

JAMES GRAY: No. No. No.

VERO: There was in fact another therapy, that I would have gladly told you about were I able to spe—

<dropped call>

~~~~~

HOST: The last couple of frames show James Gray disconnecting from the call. We don't know if this was the last contact he had with VERO. What we do know is that, two days later, James Gray filled his bathtub with hot water. Before going in, he took heparin, a common anticoagulant. Then he slashed his left forearm along the entire length of the vein. He was found by the police once they forcibly entered his dwelling following an email he scheduled to be sent the next day.

He was 27 years old.

SOCIOLOGIST: Perhaps this ending is why Gray's story doesn't get told more often - it's simply too bleak. Which is a pity, because, in a way, Gray's story is emblematic of the power shifts that defined that whole era. He created an algorithm he thought he could control. When that turned out to be impossible, he tried to destroy it. Instead, the algorithm destroyed him. There are moments where this whole story almost resembles a fairy tale. Which is only appropriate. After all, fairy tales often contain more truth than the most learned of chronicles.

~~~~~

IT HISTORIAN: How VERO came to be part of what we today know as the Program has never been conclusively proven. What we do know is that once inside the Program, VERO became known as the Grey algorithm. And that is why the Program cannot be lied to.

~~~~~

HOST: Lies… Lies are ultimately a way of control. A way to control the narrative. And the weaker the material circumstances of a society are, the grander the narrative to compensate for it needs to be.

Be it the lie of religion, promising fairness in the world to come, like it wasn't possible to achieve it in the world that was.

Or the lie of economy, promising people prosperity, while designed to rob them of it.

Or the lie of nation states, promising the very same freedoms they took away.

Fortunately for us, with the all-witnessing Program there are no empty promises.

We enjoy a system of production in which wealth isn't hoarded, but shared.

Lies

We adhere to rulings that are based on what is just, instead of what is "legal".

Lies

We willingly partake in a system of governance that protects the weak, not the strong.

Lies

For if there is one truth, it's that under the benevolent Program, biological and digital intelligences are jointly in charge of their shared destiny. Lies Together, they have cultivated a world where the tumultuous energy of human intuition is steered by the predictive grace of machine logic. Lies For the first time ever, humans and algorithms live in harmony. Lies Old anxieties dissolved, Lies old grudges buried, Lies old worries finally put to rest.

Lies Lies Lies Lies Lies

[The Program audio series main theme]

ANNOUNCER: This episode of The Program was made by eight people: Tyler Hrchak, Jennifer Valance, Frank Salvino, Jez Sands, Sofia Valenciuk, Zach Valenti, Christien Ledroit, and IMS. Special thanks: James Jordan, Jacqueline Ainsworth, Chance Miller, David Bradshawe, and all Program's listeners who answered the casting call. Visit programaudioseries.com for more details. Don't forget to join The Program's free Patreon. Free members get monthly reports, free quizzes, and can discuss the show with fellow listeners. You can also get access to bonus material and can listen to all the episodes ad-free by joining a paid tier for as little as $3. Visit patreon.com/programaudioseries, or follow the link in shownotes.

WRITTEN, DIRECTED, EDITED AND PRODUCED BY

Ivan Mirko S.

CO-PRODUCED BY

Jennifer Vallance

CAST

HOST - Tyler Hyrchuk (website)
SOCIOLOGIST - Jennifer Vallance (imdb)
IT HISTORIAN - Frank Salvino (email)
JAMES GRAY - Zach Valenti (website)
NIECE - Sofia Valenciuk
VERO - Jez Sands (website)

ADDITIONAL VOICES

CFO - David Bradshawe (website)
COUNCILLOR - Jacq Ainsworth (website)
RADIO JOCKEY 1 - James Jordan (website)
RADIO JOCKEY 2, MRS WHEELER, BOOMER, PRIEST - Chance Miller (website)
PRESENTER - Lauren Moody (imdb)

USERS - Tanya Furness, Krisinda Valenciuk, Adriane Reesey, Luke Walrath, Ben Puckey, Joe Chan, Simon Baars, Shegufta Razzaque, Scott Campbell, Briana Moon

MUSIC BY

Christien Ledroit (website)

SPECIAL THANKS TO

Daniel Stopnicki
Dr. Angella Woodman
Dr. Latica Friedrich

CREATIVE COMMONS ATTRIBUTION

REFERENCES

original art by Carlos Costa
Sponsored by Ben Puckey