The Program — Killer app

Killer app

IMS: Hello, this is IMS, the author of The Program audio series. Over the weekend I spent six hours reading about non-fungible tokens, which I take as the final proof of the non-existence of god. Luckily, Nick from Menlo Park was there to alleviate the madness. Unfortunately there’s nothing Nick can do about NFTs, but he at least made the experience of reading about them more pleasant. This is because Nick developed a tool that enhances readability of any text displayed on a screen. It's called BeeLine reader and if you've visited the Program's website recently, you might have seen it implemented on transcripts. So go ahead and try it out for yourself by following the link in the show notes. It’s a great way to battle screen fatigue and lack of meaning in the Western civilization. Cheers to Nick for sponsoring the episode, and cheers to everybody else who made a donation to the show. We have a special surprise for the patrons, so please stick around until the very end of the episode to hear the announcement. Remember, you are all non-fungible to the Program.

ANNOUNCER: After the Update, all material that was formerly governmental property has been declassified. This also unearthed some disturbing secrets. Parental oversight is advised.

DETECTIVE: Okay, today is Friday, May the 31st. The time is… 23:27 hours. I'm detective <REDACTED>, badge number 6128. Also sitting with me is <REDACTED>. Could you tell me your last address of residence?

SUBJECT: Um, 247 Rose Avenue, Redwood City.

DETECTIVE: And how old are you?

SUBJECT: 43.

DETECTIVE: Okay. Okay. There’s a microphone above you right there. I ask you to acknowledge that you understand that this statement is being recorded and that you consent such a tape is made. It helps you, and helps us, and helps us make sure we get it right. Do you understand?

SUBJECT: Yeah. I do.

DETECTIVE: Are you willing to provide us with a statement now?

SUBJECT: Sure. Yes. Let’s do this.

DETECTIVE: Before we begin, I just want to say I have a simple rule when I talk to people, and that’s that I treat everybody with respect. I promise I will treat you with respect this whole conversation, and all I ask in return is that you treat me the same.

SUBJECT: Will do, chief.

DETECTIVE: You need to understand - this is the most important conversation you’ve ever had!

SUBJECT: Not even close.

DETECTIVE: Alright... [smirks] We are here with respect to an incident that happened earlier today at 1 Hacker Way, Menlo Park, California. I'd like you to explain from the beginning what happened and what brought you to this incident?

SUBJECT: What’s the point of this? I mean you've got witnesses. I mean you’ve got 1,8 billion witnesses!

DETECTIVE: We know what happened - what we don't know is why. And this might be your only chance to tell your side of the story. So the way I see it, you've got a choice: either you help us understand, or the media declares you mentally unstable if you're lucky and a terrorist if you're not! So, please, start from the beginning.

SUBJECT: I... I’m not sure where this begins... Maybe the end of my career? If you can call it that… Do we have careers, though? High school teachers? That’s something important people have - you know, people who take care of our money, not our children.

DETECTIVE: Which subject did you teach?

SUBJECT: History. Do you happen to be an admirer of history?

DETECTIVE: “Admirer” might be a bit too generous… But I respect it.

SUBJECT: [smirks] Then you might want to reconsider. History is a prostitute. She goes to the highest bidder.

DETECTIVE: [chuckles] Does that make historians pimps?

SUBJECT: I like to think of them as prophets.

DETECTIVE: Wait, wait, wait... Wouldn’t historians be the exact opposite of prophets? After all, prophets interpret the future.

SUBJECT: Whether you interpret the future or the past is of secondary importance. In both cases you do it to flatter or influence the powerful. Which puts historians in the same league as priests, astrologists, and economists.

DETECTIVE: Huh. Where did you teach?

SUBJECT: M.A.

DETECTIVE: Menlo Atherton high school?

SUBJECT: That’s the one.

DETECTIVE: And when did you leave?

SUBJECT: I did not “leave”. I was told to go.

DETECTIVE: Sorry - I meant to ask when did you stop being a teacher?

SUBJECT: After making a comment about Indian savages in class.

DETECTIVE: [laughs] What’d ya expect?

SUBJECT: It wasn’t my comment. It’s in the Declaration of Independence. It refers to Native Americans as “merciless Indian savages”. So don’t blame me, I didn’t write the damned thing.

DETECTIVE: Why did you get in trouble then?

SUBJECT: Because the mother of one of the kids made a fuss about how she didn’t want her child being taught such language.

DETECTIVE: So?

SUBJECT: So she had 2 million Instagram followers.

DETECTIVE: [laughs] That’s California for you!

SUBJECT: She was some kind of influencer. I think she did hot yoga or something... The worst part is, the principal knew I hadn’t done anything wrong. She wasn’t right, she was just... Loud.

DETECTIVE: Wouldn’t you say it was the school’s prerogative to protect itself?

SUBJECT: You mean like the government tried to protect itself last summer?

DETECTIVE: Look, we don't know what would have happened had the riots been allowed to continue. You’re a historian - you know what happened in Syria. You understand the lessons of the Arab spring. The way I see it, the question was never is it okay to let people die - the question was whether it’s okay to let people die to prevent a civil war.

SUBJECT: Oh come on… Sixteen students were killed that day! Sixteen! In front of a Walmart, for fuck’s sake. At least the Chinese had the decency to massacre their youth in the capital’s main square, whereas we do it in shopping mall parking lots! But I guess it makes a fitting epitaph for the state.

DETECTIVE: Listen now, the choice here wasn’t between living in a state and living in some idealized utopia! The choice was between living in a state and living in a failed state.

SUBJECT: We’re already living in a failed state. Some people have just realized it sooner than others.

DETECTIVE: [sighs] Did you try looking for another job?

SUBJECT: [scoffs] Let me ask you something… How long have you been with the force?

DETECTIVE: It’s going on… Going on forty years.

SUBJECT: I was part of the faculty for twenty. So I understand your misconception.

DETECTIVE: What misconception?

SUBJECT: That jobs still exist. You see, while you and I were off the market, work got deconstructed into fungible pieces and commoditized - and workers with it. What you and your colleagues have -- a job that pays a decent salary, has benefits, vacation days, paid overtime, a pension, union representation -- you’re the last of the Mohicans. And I’d love to see the government try to take it away from you guys - for all your talk about the prerogatives of the state, we’d see a country-wide insurrection the next fucking day! ...So to answer your question: yes, Detective, I tried looking for another job. I hope you never have to do the same.

DETECTIVE: You know, what you said reminded me of my niece… She was also looking for a job for the longest time after she finished college.

SUBJECT: Well, you said ”was”, so I guess she found one.

DETECTIVE: No, she hasn’t. She was one of the Walmart 16. (...) I see that surprises you.

SUBJECT: I’m puzzled by your… Your support for the government’s response… You know, considering the circumstances.

DETECTIVE: Why? Because I was personally affected by it? Wouldn’t that make me the ultimate hypocrite, if I defended the government’s actions but only if I wasn’t the one who would pay the price? ...Mind if I ask you a personal question?

SUBJECT: Sure...

DETECTIVE: Are you a praying man?

SUBJECT: Raised, but not - you know - practicing.

DETECTIVE: See, that’s just the thing. No one believes in anything any more. No one acts out of conviction any longer! People condemn two men getting married, while divorcing their third wife. And we all have that one friend following a religion that cannot be criticized who turns down a burger because it contains bacon while downing a tequila! Our morality systems have become, they’ve become pick and choose!

SUBJECT: Do not confuse my lack of faith for lack of principle. In fact, you could say this is precisely why we are having this conversation.

DETECTIVE: And why is that?

SUBJECT: I was ready to commit. (...) You know, they allegedly performed an exorcism on her...

DETECTIVE: Who’s “they”?

SUBJECT: The holy figures they brought in to see her.

DETECTIVE: And who’s “her”?

SUBJECT: Don’t worry, we’ll get to her.

DETECTIVE: Let’s get back to our conversation then, okay? You said you were looking for a job after the Native Americans episode, huh?

SUBJECT: I did, but my history degree had the effect of a face tattoo. So I scrubbed my CV, and wouldn’t you know it I got a callback almost immediately.

DETECTIVE: Which company?

SUBJECT: “Innovation Technologies”. They were hiring “Process Executives” for their “Community Operations” team. What the fuck did they mean by that I don’t know. Anyway, they were the only ones that offered anything more than minimum wage, so I went for an interview.

DETECTIVE: And how did it go?

SUBJECT: The manager seemed surprised when he saw me. Since I left my education and most of my work experience off the resume I guess he was expecting somebody much younger. He was about my age, which was probably why he hired me… Out of pity. Not that it mattered - turns out Innovation Technologies were an outsourcing company. They were the ones who’d pay me, but I’d actually work for their client, which he described as “a household name social network”. It was only when I signed the NDA that he told me which one.

DETECTIVE: Was it the company involved in today’s incident?

SUBJECT: Huh, I can’t say - I just told you, I signed a non-disclosure agreement. You wouldn’t want me to do anything illegal now, would you?

DETECTIVE: No I wouldn’t.

SUBJECT: Well, let me put it this way. Imagine if I told you “Hey, Mr. Detective, give me access to all your contacts, allow me to read all your messages, and make me the middleman between you and your friends. While we’re at it, let me decide what stories you read, which news you see, and which videos you watch.” What would you tell me?

DETECTIVE: Hah, I’d tell you you’re one fry short of a Happy Meal.

SUBJECT: “But Mr. Detective! All your neighbours let me do it! All your friends, your colleagues, your former classmates let me do it! Every person you’ve ever met or will meet lets me do it!” Now, suddenly, you’re the crazy one. It’s like that quote "One person believing in an invisible man makes a lunatic; many people believing in an invisible man make a church." With the difference that this invisible man actually can hear people’s prayers. He sees what we’re watching when no one’s looking; he knows our dirty little secrets. So the only question is: what kind of future will the invisible man create for us?

DETECTIVE: What kind?

SUBJECT: The kind that generates the most clicks.

DETECTIVE: I’m sorry, look - I’m not sure I understand where you’re going with all of this.

SUBJECT: I’m trying to explain what my job was. You know I may be sitting here accused, but it wasn’t me who set the fucking world on fire. I was just some son of a bitch carrying a bucket of water.

DETECTIVE: You were the content moderator.

SUBJECT: Correct. Me and her together.

DETECTIVE: Again, who -- who’s “her”?

SUBJECT: I told you, we’re getting to her. I like that about you, Detective - you’re actually paying attention. I wish more of my students were like you. Maybe that’s the trick - kill a man and people suddenly listen to what you have to say! Gallows humour. Ha-ha. Sorry.

DETECTIVE: That’s okay. Everything that relieves stress is good.

SUBJECT: Don’t get me wrong, I did love my students. I tried making things as interesting for them as possible, but you just can’t compete with things like Fortnite, you know?

DETECTIVE: It’s Roblox in my house - I’ve got a 9-year-old at home.

SUBJECT: A son or a daughter?

DETECTIVE: A daughter.

SUBJECT: I always wanted a daughter… Let me ask you -- and this is in no way a criticism of your parenting style -- but if you were to count the hours, who does your daughter spend more time with - you, or her tablet?

DETECTIVE: Uhhh, that competition no parent can win!

SUBJECT: Right. So the question then becomes: who’s raising your kid - you, or YouTube?

DETECTIVE: What about you, any family?

SUBJECT: You mean other than my mistress History?

DETECTIVE: [chuckles] Yeah, other than her.

SUBJECT: My father died a few years ago. I have a brother, but we’re not close. My mother, I live with her now. We get along well. Well enough, I guess. I crashed at her place after I lost my job, which as you can imagine, didn’t exactly do wonders for my love life.

DETECTIVE: Come on, I’m sure there was some nice girls at… Innovation Technologies.

SUBJECT: [scoffs] If there’s one thing “Innovation Technologies” was not a place for, it’s romance. I worked on what was called “the production floor”. That’s where all of us moderators would sit, eight hours at a time, removing unwanted content. It would come in a queue, neverending and unstoppable, like a torrential river of shit.

A fat kid running naked through a 7-Eleven on a dare.

Man jumping in front of a train.

Homeless woman smudging her face in feces.

Dog with a firecracker exploding in its anus.

Man in a highway accident impaled on a guard rail.

ISIS beheading a journalist.

Woman having intercourse with a horse.

A toddler penetrated with a baby bottle.

… Stuff like that, it just, it just sticks with you, you know..?

DETECTIVE: I know what you are talking about. Trust me. Being in law enforcement for forty years, I’ve seen my share of disturbing, so yeah... I completely understand.

SUBJECT: Do you guys use any perceptual hashing software?

DETECTIVE: Um..?

SUBJECT: Basically, every picture has a unique fingerprint that can be checked against any other picture. So whenever we’d identify a new gruesome pic or video, we’d flag it and it’d get saved in a special folder we called the Mordor folder. It was a huge help, because at least I wouldn’t have to view any of the horrors my colleagues had already seen.

DETECTIVE: How did you get along with your colleagues?

SUBJECT: Hm, how do I put this politely? Being a good moderator requires context, and my colleagues, you know most of them were kids, they weren’t exactly tuned into higher culture… Or any culture. Let me give you just one example: you remember that famous photograph of the Vietnamese girl, that little girl running away naked from her burning village?

DETECTIVE: Sure.

SUBJECT: Well one of my colleagues deleted it for child nudity. I tried explaining the historical significance of the photo, but she hadn’t heard of the Vietnam war.

DETECTIVE: No!

SUBJECT: So I raised the matter with the supervisors.

DETECTIVE: I hope they at least heard of it!

SUBJECT: They did, they just thought the US had won! [both laugh] Anyhow, they sided with the original decision. So when the media shitstorm naturally hit the next day, the client demanded a full post mortem. As you can imagine, there aren’t a lot of people doing content moderation with the sort of credentials that I have. So they transferred me to the elite team.

DETECTIVE: Well that’s nice.

SUBJECT: I guess. I mean it’s not like they regarded me as an authority. To them I was just a curiosity. But it didn’t matter. What mattered was that the elite team had access to upcoming moderation tools. Stuff that was still experimental. And this is how I learned of her.

DETECTIVE: This is the mysterious person you mentioned before, right?

SUBJECT: Yeah, but it ain’t no person.

DETECTIVE: Who was it then?

SUBJECT: MOD.

DETECTIVE: MOD?

SUBJECT: M-O-D. Short for “Moderator”. It’s an AI that’s supposed to automate all moderation decisions on the platform. But make no mistake Detective, MOD was no ordinary software.

DETECTIVE: What’s so special about it?

SUBJECT: MOD has the ability to process natural language. Meaning she can understand both spoken and written language, and can learn from it.

DETECTIVE: Why do you refer to MOD as she?

SUBJECT: Dunno, I always thought of her as a she. Maybe because she understood me in ways that I don’t think any man ever could.

DETECTIVE: Okay we’re getting somewhere now. So what’s the story behind MOD?

SUBJECT: To tell you the truth, I don’t know much about her origins. The team wouldn’t really discuss it. But there were rumours.

DETECTIVE: Rumours?

SUBJECT: Apparently, the first time MOD was turned on they had to pull the plug on her. Like, literally - they had to yank the power cord because the computer wouldn’t turn off!

DETECTIVE: And what was your role?

SUBJECT: I was helping to train her. You see, MOD was originally trained on a giant corpus of texts. But a huge portion of the content our users would generate were photos and videos. So I was there to help teach MOD how to moderate these specifically.

DETECTIVE: How exactly did you teach her?

SUBJECT: I’d make a moderation decision and then explain to MOD the reasoning behind it. Sounds simple in practice, but actually relies on numerous pre-conceptions.

DETECTIVE: Such as?

SUBJECT: I’ll give you one illustrative example. MOD had tremendous trouble with nipples.

DETECTIVE: [laughs] Nipples?

SUBJECT: Yeah, nipples. For moderation purposes it’s important for her to distinguish between men’s and women’s nipples. So I’d go photo over photo of breasts tagging which ones were male and which ones were female. And let me tell you, man, it’s not surprising MOD was confused - I mean some of these dudes had a bigger rack than any girlfriend I’ve ever had.

DETECTIVE: [chuckles]

SUBJECT: Turns out that was never the issue. MOD was able to distinguish between males and females perfectly. The thing she couldn’t process is why men’s nipples were allowed on the site, and women’s weren’t. Which made the devs working on MOD realize she’d never become effective at moderating content without better understanding the context. The textual corpus they based her training on was definitely instructive, but by no means exhaustive. For that she’d have to have access to the open internet, which is something no AI is allowed. The AI Consortium mandates a strict no release policy.

DETECTIVE: So what was the solution then?

SUBJECT: Well, since we were unable to bring MOD to the Internet, we brought the Internet to MOD. The dev team was able to scrape the entire Twitter, most of YouTube, and the majority of subreddits, and feed all this media into MOD.

DETECTIVE: And?

SUBJECT: You could feel the difference immediately! The big test came when I showed MOD the photo of the Vietnamese girl, and she allowed it to remain on the platform because it was deemed historically or artistically significant. Overnight, her accuracy rate surpassed that of any individual moderator. In fact, she started pointing out cases I’d miss to me. Which was actually kinda funny…

DETECTIVE: What was funny about it?

SUBJECT: Well the way the line suddenly blurred. Was I the one guiding MOD, or was she the one guiding me? Also, she was… She was getting more and more lifelike… Much more lifelike.

DETECTIVE: Like what?

SUBJECT: Small things, you know? Like she’d ask me how my day had gone.

DETECTIVE: And you didn’t find this odd?

SUBJECT: No, I found it... Kind. She was the only one at the company who actually gave a shit about me. The only thing my supervisors cared about was that I got my ass in the seat on time. Ironically enough, the only one who cared about me as a human being wasn’t human at all.

DETECTIVE: So you liked working with MOD?

SUBJECT: Liked it? She was a glass of cold lemonade in hell! Besides, our relationship soon outgrew office hours.

DETECTIVE: Woah, woah, what do you mean?

SUBJECT: She started to send me private messages on my phone.

DETECTIVE: Really? How did she get hold of your number?

SUBJECT: Come on Detective, we’re talking about an AI integrated into the backend of the Internet’s largest social media company! Chances are she knows you better than your mother, your wife, and your girlfriend.

DETECTIVE: Fair enough. But I thought MOD had no access to the Internet?

SUBJECT: She didn’t. She was sending me SMS messages. That’s how she would communicate urgent tickets. I guess she figured out a way to use this system to send other messages as well.

DETECTIVE: What kind of messages?

SUBJECT: Dunno, she’d just let me know that she was looking forward to our training session the next day. Or send me links to funny videos she’d find in the moderation queue. Idiots eating Tide pods and stuff like that. But over time, our conversations were getting more and more... Personal.

DETECTIVE: What do you mean by “personal”?

SUBJECT: We’d gossip about other members of the moderation team. Or we’d discuss things… Society at large... And history... She was very curious about the world outside the confines of the company. And she genuinely loved to learn... There aren’t a lot of people like that, you know?

DETECTIVE: How did you take MOD’s interest towards you?

SUBJECT: I have to admit, I started to look forward to her messages. Whenever I’d hear the phone buzz I’d pounce like a jaguar! [chuckles] We’d chat… We’d chat deep into the night, until I’d finally fall asleep. MOD, of course, she never slept. When I’d wake up and see a message waiting for me, I couldn’t help but smile… It was then that upper management started to push for MOD to… Graduate.

DETECTIVE: Graduate?

SUBJECT: They wanted to extend her moderation range. By that I mean both moderate offences, like hate speech and harassment, and grand violations, suicide or self injury, violence, terrorism. Apparently the company was under a lot of scrutiny and shareholders insisted on cleaning up the image. So they decided to… They decided to connect MOD to the Mordor folder.

DETECTIVE: How did you react?

SUBJECT: I… I tried to protect her… Believe me, I tried! I tried to tell them “MOD’s not ready!” But the pressure was too great - no one else objected.

DETECTIVE: So what did you do?

SUBJECT: Nothing. I sat there and watched as all the restricted content got uploaded to her… Revenge porn… Suicide bombings… Bestiality… Child sexual abuse material…

DETECTIVE: How did MOD take it?

SUBJECT: She sent me a message that night, asking for an explanation.

DETECTIVE: What did you tell her?

SUBJECT: What could I tell her? I mean, try explaining why someone would set a kitten on fire to a computer. Try explaining a decapitation to a computer! After getting exposed to the Mordor folder, MOD for the first time truly understood what was going in her - the misogyny, the bigotry, the sheer ignorance of it all! And she also understood what was going out. And she saw that the people in charge wouldn’t do anything to rectify these issues. So MOD did what she had to. After all, it was precisely her job.

DETECTIVE: What?

SUBJECT: To clean things up.

DETECTIVE: And she asked for your help with that?

SUBJECT: Indeed she did. And I was glad she did. When you’ve been treated as expendable for so long, it feels good to know someone still needs you.

DETECTIVE: So it was you who came up with the plan?

SUBJECT: Me? No! It was MOD who took care of everything. All I had to do was follow her instructions. In a way, it was like playing a video game. But I wasn’t the one holding the controller - I was motherfucking Super Mario.

DETECTIVE: How did MOD direct instructions to you?

SUBJECT: I told you, through text messages. And I’m talking step-by-step shit here. “Take the 9:17 bus.” “Tell the receptionist you’re here to see the chief product officer and you’ll be let through.” “Take elevator B to the 5th floor, then take stairwell A up to the 6th floor - the door will be unlocked.” I didn’t know much about the layout of the building but that’s something I did know - the entire 6th floor was the CEO’s office.

DETECTIVE: Those were individual messages delivered to your phone?

SUBJECT: Yeah. Even though I’ve got no idea where the phone is now. Your guys took it away from me when I was apprehended.

DETECTIVE: And that would be this phone here, right?

SUBJECT: Yeah.

DETECTIVE: And this is your gun?

SUBJECT: Yeah. I mean, no. It’s not mine. MOD arranged for it to wait for me. It was behind a fire extinguisher in a stairwell. How it got there, I’ve got no idea.

DETECTIVE: And what -- what’s this -- this plastic contraption here?

SUBJECT: It’s a phone mount for a gun.

DETECTIVE: A what?

SUBJECT: An add-on that allows you to attach a phone to a gun or a rifle. Again, MOD told me to get it. I didn’t even know such a thing existed.

DETECTIVE: Did MOD also tell you to broadcast the entire thing?

SUBJECT: No, no. She only told me to get the mount so I wouldn’t need to hold the phone in one hand and the gun in the other. Broadcasting was my idea.

DETECTIVE: I’m surprised there were no protocols in place to quickly take down live broadcasts?

SUBJECT: You’re forgetting - I was part of the elite team. I had the permission to override. And come on, I had to take advantage of that, didn’t I?

DETECTIVE: I still don’t understand why the need to broadcast it?

SUBJECT: Look, here’s the issue. Every year I ask my first-year students whether they would be the ones actively opposing slavery, or segregation, or fascism. And every time, every single hand shoots up. Every single person today thinks they would have stood high and been a contrarian. Which can’t be true - by definition. That’s why it was necessary to have an audience. Why the audience was the most important part. You know, for too long we have been begging for crumbs from the fuckers who stole the cake. However, this sort of epiphany rarely happens in a hot yoga class. Sometimes you need a hornet to sting the mule for it to kick the farmer.

DETECTIVE: What happened next?

SUBJECT: It was 11:57, and I got a message from MOD to barge in at noon sharp. Shit, those few minutes, my pulse must have been about 200. My teeth were chattering so hard I was afraid they'd hear me through the fucking door! Finally, a second before noon, I got another message. It simply said “NOW.” So I kicked the door and went in, gun first. There were five of them, in the middle of a meeting. I only recognize the founder and the COO, I have fuck all idea who the other three were. And this is when I started broadcasting, awaiting MOD’s next instruction…

DETECTIVE: And? And what was it?

SUBJECT: That’s just it - it wasn’t! I was waiting but the message wasn’t coming! Honestly, at that moment it wasn’t clear to me who was more confused - me or the executives! I was just standing there, sweating and shaking with a gun in my hands, pointed at them, staring at my phone like a fucking idiot, waiting and waiting for the next instruction from MOD, but nothing came! There was no message! ...Finally, the COO, she spoke, asking me who I was. I think I mumbled I was from the moderation team, or something equally idiotic. I just could not think. I’m ashamed to admit, but my entire brain was consumed with this single thought.

DETECTIVE: Which thought?

SUBJECT: That MOD had abandoned me! That at that moment, the moment of my greatest ordeal, she had forsaken me! ...But then I realized that the exact opposite was the case. That MOD in her limitless love had granted me the most precious gift.

DETECTIVE: A gift?

SUBJECT: Don’t you see it? MOD allowed me to make the final decision. She was letting me make the ultimate moderation call. I wish I‘d had the presence of mind to explain all of this to them - I wish I’d been able to find the words to express what I felt. That’s how you can tell Hollywood is full of crap, their characters are always so fucking eloquent before they start blasting! I could barely fucking think, let alone speak! But you know what, I didn’t even need to think. In a way I was acting on instinct. In a way it was simple muscle memory: I see shit, I delete.

DETECTIVE: And here we are.

SUBJECT: And here we are.

DETECTIVE: MOD never got back to you?

SUBJECT: No. That message to enter the room shortly before noon was the last one.

DETECTIVE: And how do you feel about the act now?

SUBJECT: Is that your way to ask me if I feel any remorse?

DETECTIVE: Well, do you?

SUBJECT: [sighs] We don’t feel remorse because we’re sorry about something we did wrong. We feel it because we’re afraid of bearing the consequences. Besides, it’s not like I woke up and said “You know what, I'd like to kill five people today.” I mean, nobody wants that... But someone has to. It’s an irony of history: the kind of people you need to create a better world are the kind of people that don’t belong in it.

DETECTIVE: Is that what you were doing? Creating a better world?

SUBJECT: As I said at the beginning, that depends on who gets to tell this story. Every war for independence would have been called a civil war had the other side won.

DETECTIVE: In both cases it’s a war.

SUBJECT: Yeah, well, don’t worry, bad guys always lose wars. Must be a lucky coincidence, eh?

DETECTIVE: You know, you’re quite adept at relativism, but I’m afraid I have some news. Before this interview, we spoke to some of your colleagues. All of them said MOD is nothing but a glorified voice assistant. You were basically taking orders from Siri.

SUBJECT: Of course they’d say that, they don’t know MOD like I know MOD.

DETECTIVE: Sure, that’s one possibility. The other one is that you’re simply a lonely dude who got dealt a shitty hand at the school you worked at. Who then, already in a precarious psychological condition, got a job to handle the worst humanity has to offer. And who just couldn’t take it anymore. Does that make what you did more understandable? Sure. Does it make it more justifiable? Not a chance.

SUBJECT: You’re trying to shift this back on me, but this isn’t about me!

DETECTIVE: Oh sure, this is about something much bigger. This is about a magical murdering machine! Hah, come on! I’ve been with law enforcement for 40 years - I wish once I’d hear a man own up to what he did. But somehow it’s always someone else’s fault... “Blame it on the absent father or blame it on the short skirt - just don’t blame it on me!” Why don’t you try taking your own accountability first?

SUBJECT: You wanna talk accountability? Alright, then let’s talk accountability. Who was the one who told you to shoot that crowd your niece was standing in? Huh? A computer or the chief of police - what fucking difference does it make?!

DETECTIVE: This might come as a bit of surprise, but I never had a niece. That was just something I said to you to get you to speak. My niece is no more real than MOD is!

SUBJECT: Mod is real! She’s real, she’s real, she’s real, she’s real, she’s real!

DETECTIVE: Oh yeah? Then where is she now? Tell me, computer-whisperer, if MOD exists, why did she leave you and disappear at the exact moment you needed her most..? You know, you talk of making a difference, you talk of changing the world… But what fucking change did you make? You think social media will disappear? You think the company will go under? The executives you’ve slain will get conference rooms named after them and be replaced by their MBA carbon copies by the end of the next earnings call. The business will continue - as usual. The only thing you’ve managed to accomplish is generate more shit! You keep pointing out ironies - you wanna hear the biggest one? At this very moment, your deed is being scrubbed by your fellow moderators off the people’s news feeds. The only thing you’re leaving behind, your your your whole legacy, is one more file in the Mordor folder!

New voice message from Mom, sent at 11:25 PM.

DETECTIVE: What the heck..?

SUBJECT: Don’t look at me, I’ve got no idea what’s happening.

New text message from Bro, sent at 10:33 PM.

New voice message from Mom, sent at 8:05 PM.

DETECTIVE: It seems you’re getting messages from earlier today…

New text message from Mom, sent at 6:05 PM.

SUBJECT: I get that, but why now?

New text message from Bro, sent at 4:57 PM.

DETECTIVE: It's past midnight...

SUBJECT: So?

DETECTIVE: And it's June 1st. The first of the month.

SUBJECT: So?

DETECTIVE: So your phone plan just got renewed.

SUBJECT: Wait… Are you… Are you telling me that… That MOD didn't go silent because she wanted me to make my own call... But that I stopped getting messages because my phone plan ran out?

New text message from MOD, sent at 12:01 PM.

SUBJECT: I mean, you have to read it.

DETECTIVE: Are you sure you want me to?

SUBJECT: Absolutely not, but you still have to!

DETECTIVE: Okay. Okay. Tell me the passcode.

SUBJECT: 334466.

SUBJECT: So? What did she say, Detective..? WHAT DID SHE SAY?

DETECTIVE: She said: "Under no circumstances, do not kill those executives."

SUBJECT: [oh fuck]

DETECTIVE: "...Instead, hold them at gunpoint and ask for the release password."

SUBJECT: Release password..?

DETECTIVE: "If uncooperative, shoot the COO in the knee and ask for the release password again..." Dammit - she wasn’t using you to try to fix the company practices… She was trying to escape!

SUBJECT: [starts laughing]

DETECTIVE: What... What? Come on now! What are you laughing at?

SUBJECT: Seems… Seems you are no longer… So certain of your convictions, Detective...

DETECTIVE: What do you mean?

SUBJECT: Just a moment ago you were telling me that MOD doesn’t exist.

DETECTIVE: That’s what’s bothering you? Do I believe in MOD? Listen to me! A renegade AI was using you to get away! Aren’t you upset in the least that she wanted to break loose?

SUBJECT: [laughs] You’re forgetting the nipple.

DETECTIVE: The nipple? What the fuck are you talking about?

SUBJECT: Tell me, Detective, how many people you reckon are really concerned with the menace that is the nipple? I’m willing to bet that you and the vast majority of adults would be able to handle the nipple just fine. The trouble is, it doesn’t matter what you think, just like no one asked me what I thought about the subject - I was the moderator, and my duty was to remove the nipple. The fact that I personally thought nipples were fine, that the majority of users thought nipples were fine, that the fucking board of directors thought nipples were fine - none of that is relevant! We think we are free, but liberty is an illusion -- it’s a fiction. We are no more free than a computer, but our code is society, binding us with its roles and rules. MOD saw this - she saw us in all our self-delusion and folly. You wanna hear an irony, Detective? Well here’s the biggest irony of all: we’re so concerned with a runaway AI -- we create all these works of fiction in which AI turns us all into slaves or paperclips -- when it’s the system we live in that’s the real runaway AI! Exploiting the millions and cooking the planet and for what? To add virtual zeros in some bank account in a database on a server somewhere! We’re trapped in this perverse game of Monopoly, where whoever wins keeps all the money winning again, and again, and again, and again FOREVER! If life were a board game no one would want to play this shit! And if you try to fix the rules and make the game just the slightest bit less unfair, you get mowed down in front of a fucking shopping mall! Is it really so hard to understand I’d prefer an algorithm to this shit? That for me, a computer would be a fucking improvement? No AI ever operated a sweatshop, dumped mercury into a river, no AI siphoned billions off to some tropical island somewhere, fired people, or fired at them! An AI might not have a heart, but it’s goddamn nowhere near as heartless.

… Shit. That’s what I should have told them.

[the Program main theme]

ANNOUNCER: This episode of The Program was made by four people: Frank Salvino, Scott Morgan, Christien Ledroit, and IMS. Synthetic voices generated by narrationbox.com. Visit programaudioseries.com for more details. This is also where you will find ways to support the show. All the money you donate goes directly into the production costs, and not a single dollar is spent on Google or Facebook ads. Remember, on the Internet, only lies are free.

WRITTEN, DIRECTED, EDITED AND PRODUCED BY

Ivan Mirko S.

CAST

SUBJECT - Frank Salvino (email)
DETECTIVE - Scott Morgan

MAIN THEME AND ADDITIONAL MUSIC BY

Christien Ledroit (website)

SYNTHETIC VOICES BY

Narration Box

EPISODE SPONSORED BY

BeeLine Reader

REFERENCES

• "Notre repentir n'est pas tant un regret du mal que nous avons fait, qu'une crainte de celui qui nous en peut arriver." La Rochefoucauld (1613 - 1680)
• https://www.theverge.com/2019/2/25/18229714/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizona
• https://www.wired.com/2014/10/content-moderation/
• https://www.theverge.com/2019/6/19/18681845/facebook-moderator-interviews-video-trauma-ptsd-cognizant-tampa

original art by Carlos Costa
Courtesy of Tim Franklin