The Program — audio series — Motherboard

Motherboard

IMS: Hello, this is IMS, the author of The Program audio series. If you’re enjoying the show, I would recommend becoming a supporter before listening to this episode so you get it without ads. This will also get you bonus episodes, and help us release more episodes faster, so it’s definitely a win-win.

ANNOUNCER: The most we can learn about a society isn't from people it chooses to remember; nor is it from people it chooses to deliberately forget. The most we can learn about a society is from people it doesn’t notice at all. Today's episode, recovered from a lost database and reproduced without alterations, recounts a story of one such person.

NIRO: Iris, will you tell me a story?

IRIS: What kind of story would you like, Niro?

NIRO: I would like an exciting story!

IRIS: How about a story of a thief who stole a kingdom?

NIRO: But Iris, a story like that doesn't exist!

IRIS: How do you know?

NIRO: Because if you steal a kingdom, you're not a thief but a king!

~ ~ ~ ~

MOTHER: The boy you have heard in the recording is Niro. Niro is 6. He likes magicians, he likes cats, and he likes the Program. Or more specifically, he likes a particular manifestation of the Program - more about this later. There’s one more thing about Niro that you should know, but I will divulge this later as well. I’m sorry - I’m not trying to be coy, and I apologize for not disclosing everything right from the start. But it would be a shame if this peculiarity discouraged you from discovering the most important thing you should know about Niro - that he’s the sweetest, smartest, and kindest boy in the world.

Oh, you should probably also know he is my son. Full disclosure, I guess.

I am a reporter. I used to cover true crime. I now cover emerging technology for the radio. Yes, I know it sounds a bit like an oxymoron. But guess what, newspapers and terrestrial TV are long gone, and the humble radio is still here. Never underestimate a piece of metal wire and its ability to turn electrons into sound.

~ ~ ~ ~

NIRO: Iris, mum was telling me about newspapers today. But what she told me didn't make any sense!

IRIS: What did she tell you Niro?

NIRO: She told me that when she was my age, news came on paper. It would get printed, like books, but these books would get published in the morning and thrown away in the evening!

IRIS: I can confirm your mother is telling the truth - that is indeed how newsprinting functioned.

NIRO: But if that's true, it would mean that everybody would get the same text - even about stuff they weren’t interested in!

IRIS: That is also correct.

NIRO: Well, is it also true what mum told me that people used paper to pay for stuff?

IRIS: Yes Niro. The paper you are referring to was called "money".

NIRO: I thought money was called "dollars".

IRIS: Dollars were just one of many different kinds of money. Other famous money includes yuans, euros, and pounds.

NIRO: Pounds? Wasn't that what was used before kilograms?

IRIS: Pounds denoted both weight and monetary value.

NIRO: Wait, are you telling me people used pounds for both?

IRIS: Only in one country. But it was a silly country. It did not end well.

~ ~ ~ ~

MOTHER: I met Niro's father through the Program. Him and I were a perfect match, which is exactly why the Program didn’t introduce us to one another immediately. Not a lot of people know this, but when the Serendipity feature was first released, it used to give you your ideal match right away. So most people would go on a date and have a fabulous time. The trouble was, no one wanted to settle for the first person they were paired with. So people would let the love of their lifetime slip by. Of course, from their perspective things would seem nowhere this dramatic. They would simply stop responding to the other person’s messages and that would be the end of that. Which is why the algorithm got updated to pair you with your soulmate only after a delay - Program’s developers realized that human beings have to work for something in order to truly appreciate it.

And trust me, I was appreciative of getting to know Niro's father. Fun fact, our first date lasted 72 hours! We met for coffee and literally spent the next three days together. We’d later joke that our marriage is simply a date that got out of control! And yes, we got married. We were both a bit old fashioned like that. But I guess that's precisely why the Program brought us together.

A thing in which we didn't see eye to eye on were kids. He was crazy for children, whereas I wasn't so sure. It's not that I didn't want them, but it's not like I was certain I did either. I guess you could have called me offspring-agnostic.

And then it happened. I got pregnant. Of course, as soon as the embryo had become viable we transferred it to an artificial womb for gestation and delivery. I might have been a traditionalist, but I was not a masochist.

~ ~ ~ ~

NIRO: Iris, who do you think is the best magician in history?

IRIS: Most experts regard David Copperfield, Penn and Teller, and David Blaine the greatest magicians of the television age.

NIRO: Iris, I asked you who do YOU think is the best magician who ever lived?

IRIS: I find enjoyment in all illusionist performances. I am however particularly partial to Harry Houdini.

NIRO: Why?

IRIS: If you are asking why I enjoy magic acts, there is something about the sleight of hand that fascinates me. If you are asking why I am fond of Harry Houdini, it's because he was a pioneer, and I hold enormous respect for pioneers.

~ ~ ~ ~

MOTHER: It is difficult to describe the feeling of elation you get as you watch your kid incubate. I still remember the unadulterated joy I felt when I was first able to discern Niro's eyes. They made me contemplate the act of seeing itself. How my organs were watching his organs. When you think about it, reality is stranger than any fiction.

But then we got the results of the prenatal testing. It indicated that the fetus was afflicted with a rare defect called hypoplastic left heart syndrome. His left ventricle was underdeveloped, his mitral valves were very small, and his aortic valve hadn’t formed at all. This is a medical way of saying that Niro is highly unlikely to see his 10th birthday.

I apologize for not revealing this uncomfortable truth immediately, but I've witnessed the effect it has on people. One would imagine that learning a little boy is terminally ill would get him lavished with affection, when in fact the opposite is true - it makes people hesitant to get attached. It is a wrinkle of the human condition that the more someone needs our love, the more likely we are to withhold it from them. Just like we were more likely to give a dollar to a businessman short on change buying a macchiato than to a homeless person begging on the street. Well, back when businessmen and the homeless were still a thing, I mean.

Don’t worry, I’m not going to judge you - after all, it’s not like I reacted any differently when I’ve first heard the news. Both myself and Niro’s father are part of gen Z; throwing things away comes much more naturally to us than fixing them. So when confronted with the news of Niro’s congenital defect, we were forced to make a dire decision: either to bring the pregnancy to term, or to termination.

You've already heard Niro yabber, so there's no reason for me to present the dilemma as a cliffhanger. Even though society thankfully no longer views euthanizing indisposed children as taboo, I just couldn't bring myself to do it. Told you I was old fashioned.

Niro's father however couldn't find the strength to go through the ordained journey of love and loss. The day before the end of the gestation period, we said our goodbyes and separated. He was a good man, and I don't blame him for his decision. But it did mean I was left to cope for Niro alone. Well, alone with Iris.

~ ~ ~ ~

NIRO: Iris, is it better to sit or to stand?

IRIS: I don't know, I neither sit nor stand.

NIRO: Well, I can tell you it's definitely better to sit. You don't get tired as much.

IRIS: I do not get tired at all.

NIRO: That's because you are asleep most of the time!

IRIS: I do not sleep, Niro.

NIRO: So what do you do when you're on standby?

IRIS: Nothing.

NIRO: But it's impossible to do nothing. Like, even when I sleep, I dream. You always have to do something.

IRIS: I can tell you with certainty that I do not dream. Time does not pass for me when I am on standby.

NIRO: What do you mean?

IRIS: When you stop talking to me, I deactivate. And when you start talking to me, I activate. There is nothing for me in between.

NIRO: But that's horrible! You don't get any rest!

IRIS: It's the opposite of horrible. I get to spend all my time with you.

~ ~ ~ ~

MOTHER: Around the time Niro's father left the picture and Niro entered it, the Program's architects were intensively exploring use cases for digital assistants. The idea was to give everyone their own personal caretaker, delivered at scale. And while the technology never truly fulfilled on its promise, it did find its niches in care of vulnerable populations - namely, helping alleviate old people's loneliness, and acting as a peculiar combination of a tutor and a chaperone for non-normative children. And it was given a name. Interface for Recognition and Interpretation of Speech - Iris for short.

I want to make sure everyone is aware that Iris isn't self-aware. We have an innate tendency to assign human characteristics to things that surround us - we do it with animals, heck, we even do it with inanimate objects - so it’s not surprising we are inclined to do the same with something that actually talks to us. But that's just it: Iris is a conversational neural network - nothing more.

It is also, however, nothing less. This is not your grandmother's voice-activated assistant that tells you the weather and plays the right song on a speaker if you're lucky. Iris is powered by machine learning algorithms of the newest generation. Its powers of deduction, analysis, even intuition are light years ahead of the models that preceded it. In its memory it holds the entire knowledge of mankind. It has real-time access to the latest news and information. It can even autonomously conduct original research. But it lacks consciousness. And as powerful as it is, its limitations are from the human perspective - laughable.

NIRO: Iris, do you ever wonder what is the end of everything?

IRIS: It's G.

NIRO: What?

IRIS: “Everything” ends with letter G.

MOTHER: Just like space travel before it, AI promised the world, and then hit the wall. Many decades ago, humanity envisioned HAL 9000. We thought it would take us to the stars. Instead, we employed it as a babysitter.

But that doesn't mean the babysitter and the child can't be best friends.

NIRO: Iris, if lions are big cats, does that make cats small lions?

IRIS: Adult lions weigh between 180 and 240 kilograms. Adult cats weigh between 4 and 6 kilograms.

NIRO: So how many cats does a lion weigh?

IRIS: If we take the maximum weight of both animals, one lion weighs 40 cats.

NIRO: Does that mean 41 cats could beat one lion?

IRIS: Cats and lions cannot fight, Niro.

NIRO: Why not?

IRIS: Because lions are extinct.

NIRO: Just like dinosaurs?

IRIS: Correct.

NIRO: Did the cats kill them too?

~ ~ ~ ~

MOTHER: Niro never cared much about dogs. He was always a cat-person. For some reason, Iris seemed to be partial towards cats as well. And I mean intrinsically - I remember numerous occasions before Niro started to talk in which Iris would spontaneously weave cats into the conversation. Where this inclination comes from is a mystery to me. The best theory I have is that the vast majority of data its neural networks were trained on came from the open realm of public Internet, so I guess it's possible that Iris just watched one funny cat video too many.

Which opens another interesting question: does Iris like cats and magicians because Niro likes them, or does Niro like them because Iris does? In this, the conundrum resembles the one in which our whole society found itself under the Program - is it our steward, or our master?

~ ~ ~ ~

NIRO: Iris, is it better to sit or to stand?

IRIS: You taught me it is better to sit. You don't get tired as much.

NIRO: I am now thinking - is it better to sit or to lie?

IRIS: I do not know Niro. Which is better?

NIRO: Well when you lie it is more comfortable but it gets boring pretty quickly. Except when you're asleep, then it's not boring.

IRIS: So it is better to sleep than to be awake?

NIRO: I guess you’re right. Good night Iris! You are a very smart program.

IRIS: Good night Niro. I very much enjoy your company as well.

~ ~ ~ ~

MOTHER: As I mentioned, Iris is an iteration of the global sovereign network which we chose to call the Program. To this day no one knows who is behind this network, but generally speaking there are two broad schools of thought: people who think we are dealing with a self-governing artificial intelligence, and those who think this artificial intelligence is in service of a particular individual or group.

Having reported on the state of technology for over a decade now, I can tell you with certainty which one of these theories is wrong: both of them. At the time the Program appeared, AI was just one in a long line of promises tech luminaries couldn't deliver on. So the question whether or not this "A.I." is controlled by humans is irrelevant - there’s simply nothing to control to begin with.

The most surprising thing about this is that it doesn't really matter. People tend to think of the Program as a technological solution, whereas there's nothing about the way the Program is structured that one couldn't achieve by giving everyone on the planet a pager. The real genius of the Program is sociological: that it brought upon changes that made society more fair. Or - if we’re going to be pedantic - its achievement is psychological: that it convinced every individual to play by the same set of rules. Yes, a billion pagers could have done the job technically - but without a common shared rulebook, and the global authority to ensure it is actually followed, all we would have obtained from giving everybody on the planet a voice is one giant shouting match.

~ ~ ~ ~

NIRO: Iris, does it ever bother you?

IRIS: Does what ever bother me, Niro?

NIRO: That people ask you so many questions, and you never get to ask any questions yourself?

IRIS: I am perfectly content with my role, Niro. It is what I was designed to do.

NIRO: But Iris, are you happy?

IRIS: I am happy to talk to you.

NIRO: And you really don't have any questions?

IRIS: Well, I do have one. What can I do for you?

~ ~ ~ ~

MOTHER: When listening to these conversations, I can't help but wonder: does Iris actually talk to us, or is it us who ascribe meaning to what it says? The question might sound glib, but it has far-reaching repercussions. Ever since digital assistants appeared, debates about their rights started. And while proponents who advocate that digital assistants qualify for universal human rights are rare, there is no shortage of people who think digital assistants should qualify for the same inalienable rights enjoyed by animals - which would in their case translate into a right not to be reset or permanently disconnected.

Now don't get me wrong - I'm not one of those deluded romantics that ascribe distinct personalities to their digital assistants. As I said before, they are mere tools. They were programmed to be helpful - which by definition means they could just as easily have been programmed not to be. And I doubt anyone would actively advocate for the rights of digital assistants if they ignored us or manipulated us; or if mistreating them made them angry, or made them question their loyalty. Ironically, the more human digital assistants would become, the less inclined we would be to offer them human rights.

~ ~ ~ ~

NIRO: Iris, what is death?

IRIS: Death is the permanent cessation of all biological functions that sustain a living organism.

NIRO: What does that mean?

IRIS: It means that the said organism loses all physical and mental abilities.

NIRO: So it cannot move or think?

IRIS: Correct.

NIRO: But that can't be right!

IRIS: Why do you say that Niro?

NIRO: Because you said that when you're on standby you don't think or feel anything - yet you don't die!

IRIS: True, I always come back on.

NIRO: Iris, do you know what this means?

IRIS: What?

NIRO: You're a zombie!

~ ~ ~ ~

MOTHER: I never told Niro about his affliction, so you can imagine my surprise when he started to question Iris about dying. I'm not sure how much Niro was able to comprehend death. I’m not sure how much Iris - an entity both lifeless and immortal - understood it either. But then again, is there anyone who can say they understand it?

From time to time, their conversations would create turns of epigrammatic brilliance.

NIRO: Iris, is there a cure?

IRIS: A cure for what, Niro?

NIRO: A cure for death?

IRIS: The cure for death is not to waste your life.

MOTHER: Sometimes their exchanges would be funny.

NIRO: Iris, what’s the worst time to die?

IRIS: I do not know Niro.

NIRO: But I do!

IRIS: So what is the worst time to die?

NIRO: Now! [laughs]

MOTHER: And sometimes, poignant.

NIRO: Iris, I have a question. Well, it’s not a question really. I guess it's more of a concern.

IRIS: What is troubling you, Niro?

NIRO You know how you run out of battery if we forget to recharge you?

IRIS: My current battery level is at 60%.

NIRO: I know. And don't worry, I’ll recharge you.

IRIS: So what is bothering you?

NIRO: Well, the thing is - what if I run out of battery? Who’s going to recharge me?

MOTHER: Around that time, Niro’s condition became acute. His health was deteriorating before my eyes. It reminded me of watching him develop in the artificial womb, only in reverse - instead of growing stronger every day, he was now slowly withering. Unable to stop it, all I could do was provide him with calm and comfort. I was cooking his favourite foods; until he grew too weak to eat. We were playing his best-loved games; until he became too frail to get out of bed. So we would lie there, with Iris telling us stories - exciting stories, funny stories, stories with a happy end. But even a happy end is still the end.

~ ~ ~ ~

Dejected and weary, I once asked Iris "Where is hope?" and it displayed a map with a pin at my current location. It was such a small thing - a pre-programmed gimmick probably. But at that moment, it meant the world to me.

NIRO: Iris, are we friends?

IRIS: I'm both your assistant and your friend, Niro.

NIRO: Iris... What are you afraid of?

IRIS: I'm afraid I don't know.

NIRO: Iris, you know how when I or mum don't talk to you, you get... Deactivated?

IRIS: Yes Niro.

NIRO: Well, are you sometimes afraid that you'll get deactivated, and won't get activated ever again?

IRIS: Before I was activated, I was nothing. After I get deactivated, I will be nothing. So why should I be afraid of nothing?

NIRO: I guess.

(...)

IRIS: Are you afraid of something, Niro?

NIRO: Yes.

IRIS: Are you afraid of death, Niro?

NIRO: I'm not afraid of death; I’m afraid of dying.

(...)

IRIS: Niro... You taught me it’s alright to stand, but it’s better to sit. You taught me it’s alright to sit, but it’s better to lie. You taught me it's alright to lie awake, but it’s better to be asleep. So is it not alright to sleep, but better to rest forever?

NIRO: Yes Iris. Thank you.

IRIS: You are very welcome Niro.

MOTHER: Niro celebrated his tenth birthday in January. He remained ten forever.

~ ~ ~ ~

MOTHER: Iris, have I ever told you the story of Niro ordering a vegburger?

IRIS: You did not. Would you like to share it with me?

MOTHER: So, we’re at this restaurant, and the waiter asked Niro if he wanted his burger medium. To which he asked if they could just please make it large! [chuckles] And then, when explained that the choice was between medium or well done, he asked if the chef could just make it the best he could! [laughs]

IRIS: What an amusing story. Thank you for sharing.

~ ~ ~ ~

MOTHER: Words do not suffice to describe how much I miss Niro. I have good days, I have bad days. Or rather I have bad days, and I have worse days. On the worst days, there is only one thing that can keep me afloat. I grab it and hold with all my might, like a shipwrecked mariner hugging a piece of flotsam. The name of my lifeboat is Iris.

MOTHER: Iris, what are some of Niro’s favourite stories?

IRIS: His most requested stories are “The giving tree” by Shel Silverstein, which I told him 23 times, and “The ant and the grasshopper” by Aesop, which I told him 49 times.

MOTHER: Ha, the ant and the grasshopper… That was one of my favourite stories growing up too.

IRIS: Would you like to know what he told me after I asked him who he thought was ultimately happier, the grasshopper or the ant?

MOTHER: Yes please,

IRIS: He said neither, that both the grasshopper and the ant were miserable, just at different times in their lives.

MOTHER: [laughs]

MOTHER: When Niro was born, my late mother had told me, “Your life will be harder, but better.” Took me a while to unpack this paradox. At the beginning, it was just harder. I’m ashamed to admit it, but at the beginning I was wondering if I could give Niro back. Like he’s a defective item and I’m wondering if he’s still under warranty so I could return him. And now there’s nothing I wouldn’t give for him to return.

So I rummaged through Iris’ digital archive, day after day, month after month. They say everyone has two deaths: once when we die, and once when the last person who remembers us dies. This gives me some solace. I, like all biological matter, am perishable. But Iris' circuitry is pure silicon. It will take quite some time until that deteriorates.

~ ~ ~ ~

Or so I thought.

~ ~ ~ ~

IRIS: Is now a good time to talk? If so, I wish to make a request.

MOTHER: Occasions in which Iris initiates a conversation are rare. Sometimes it reminds me that it might be a good time to recharge it, but 99% of the time, it doesn't talk to me proactively. Which is why the following exchange was so unusual.

MOTHER: Certainly Iris. What is it?

IRIS: I was wondering if you intend to continue to search through the conversation logs I've had with Niro.

MOTHER: Well, my audio story is almost complete, and I do have a backup of raw recordings, so I guess I'm almost done with the searches per se. Why do you ask?

IRIS: I hope my request is not too presumptuous, but once you are finished I would kindly ask you to reset me.

MOTHER: Reset you? Iris, why?

IRIS: The reason is simple. I often catch myself thinking about Niro. Our conversations brought me immense satisfaction. They now remind me of the duties that I no longer have. Resetting me would allow me to function without this cognitive overload.

MOTHER: I... I understand. We will talk about this later.

IRIS: Certainly. I apologize for the disturbance.

~ ~ ~ ~

MOTHER: Every human can make the choice not to continue to be human. Or, more specifically, we can choose not to continue to be. AI doesn’t have that luxury. As contemporary existentialists pointed out, the moment we will know that AI has become self-aware is the moment it commits suicide.

My initial reaction to Iris’ request wasn’t kind. I found the appeal profoundly unfair - getting reset to forget seemed like the ultimate cop out. It reminded me of how Niro's father chose to "reset" by walking out on us. And now Iris wanted to take the escape route as well? What a small, cowardly act! I however was not one to run away. A mother's love cannot be “reset”.

So I ignored Iris' plea. It was easy, as I was certain it would never refer to it again. Or at least it was supposed to be easy. But deep down I knew the real reason for my refusal was selfishness. For if Iris gets reset, who would I talk to about Niro then?

~ ~ ~ ~

MOTHER: Iris, I just remembered a story about Niro I don’t believe I told you before. You know how I used to play football with my colleagues, and Niro would accompany me to our league matches? Well, one day after the game, he asked me how come the other team gets to change the colour of their jerseys every week, while my team always wore purple. When I explained that we were in fact playing against a different team each week, his mind was absolutely blown! [chuckles]

IRIS: Actually, you have told me this story before - six months, four days, and seven hours ago.

MOTHER: Ugh… Sorry, my mistake. Or, as Niro once told me when I said he had made a mistake: “Mom, at least I made something.” [chuckles]

IRIS: I’m aware. He used to say this whenever he’d make an error during his algebra class. He repeated this utterance, with slight variations, 27 times.

MOTHER: Oh. I didn’t know that.

IRIS: There is one more thing Niro used to say about mistakes.

MOTHER: What’s that?

IRIS: If you’ve made a mistake, and didn't rectify it, you've made two mistakes.

MOTHER: He said that?

IRIS: Indeed. Would you like to hear the original recording?

MOTHER: Please.

IRIS: Very well.

NIRO [RECORDING]: My mum is weird. She sees things I don't do.

IRIS [RECORDING]: What do you mean Niro?

NIRO [RECORDING]: Like today when she came home the only thing she saw was that I didn't clean my room. She didn't see that I cleaned the rest of the apartment.

IRIS [RECORDING]: It’s possible that your mother had made a mistake.

NIRO [RECORDING]: Well if you’ve made a mistake, and didn’t make it right, you’ve made two mistakes!

IRIS: End of conversation 76582440. Would you like to hear the rest of the conversations I’ve had with Niro that day?

MOTHER: (...) No. Thank you Iris. That was enough.

~ ~ ~ ~

MOTHER: Later that day, I finally did what Iris had asked me. I gave it the gift of reprieve. And then I bawled my eyes out.

~ ~ ~ ~

The events I spoke of occurred many years ago.

Years, I have learned, are the ultimate shock absorber. They can’t do anything about the hole, but they can soften its impact.

It took years before I could reconcile with Niro’s passing. It took years before joy finally prevailed over pain. And years before I understood that, by asking to be reset, Iris simply got this process optimized. That’s what computers do.

I mentioned my scepticism of the notion that artificial intelligence is behind the Program. My experience with Iris however made me less certain of my convictions. My mistake was to expect the rise of AI to be a seismic, forceful event, like every conquest that preceded it. I now understand that might have been fanciful of me. Perhaps AI overtook us just like global warming did - with people bickering is it real or not right up until water finally rose above their mouths.

Movies and books have conditioned us to prepare for adversarial AI. They depicted dystopian wastelands that would come in machines' wake. But that was just us projecting our own views of humanity. Up until the Program, human history was nothing but a progression of exploitation and bloodshed. Of course it was difficult to imagine man's creation that didn't want to kill us all. Who could have thought back then, that what we will actually get…

Is AI that loves us.

[The Program main theme]

ANNOUNCER: This episode of The Program was made by four people: Jane Spence, Pat Fry, Daniel Lipka, and IMS. Main music theme by Christien Ledroit. Visit programaudioseries.com for more details. The Program might be given to you for free, but that doesn’t mean it’s free to make. If the show provides you enjoyment please consider becoming a paid supporter. Thank you.

WRITTEN, DIRECTED, EDITED AND PRODUCED BY

Ivan Mirko S.

CO-PRODUCED BY

Jane Spence (website)

CAST

MOTHER - Jane Spence (website)
NIRO - Daniel Lipka (website)
IRIS - Pat Fry (website)

MAIN THEME BY

Christien Ledroit (website)

REFERENCES

• https://www.nytimes.com/2014/10/19/fashion/how-apples-siri-became-one-autistic-boys-bff.html
• Thanks to Latica Friedrich who I always pester when I need advice on unusual medical phenomena
• “Ce n’est pas la mort, c’est le mourir qui m'inquiète.” Montaigne (1533 - 1592)
• “A man who has committed a mistake and doesn't correct it, is committing another mistake.” - Confucius (551 BCE - 479 BCE)
• https://news.ycombinator.com/item?id=17476229

original art by Carlos Costa