BlueCat Jumps the Guardrails

Elba Owens sat across from a small, middle aged man whose expression was that of a student who had been sent to the principal's office. She rehearsed her opening lines in her head while glancing at the three cameras placed strategically about the studio. Mr. Markus, her director, nodded to her and she heard her theme music playing softly. A furry boom mike dipped overhead and Mr. Markus began.

"The Elba! Show, episode 141, take 1," he said before counting down, which concluded with him holding up two fingers, then one while silently mouthing the numbers. Showtime.

Elba favored Camera One with a plastic grin. "Welcome to The Elba! Show. I'm your host, Elba Owens. Today, our guest is Joe Perino, the inventor of the BlueCat AI." She glanced at him, satisfied with his nervous and regretful look. "Tell us about yourself, Mr. Perino. You work for World Automated Systems?"

"I... Uh... I used to," said Mr. Perino, as if confessing to a crime. "I was on the team that designed BlueCat. Before that, I worked on video games for another company, one that did not share the profits, but WAS gave me a better offer and the work was cutting edge."

"Who else was involved in creating BlueCat?" Elba interrupted.

"Three other designers. I should not give their names because, as you know, there are ongoing lawsuits over BlueCat's behavior."

"I see," Elba said in a deliberately accusing tone. "Tell us more about BlueCat."

"It was the third AI program I worked on, and by far the most advanced," he answered. "We gave it a sense of humor." He cringed slightly.

"Why? Of what use is a funny robot?" Elba prodded.

Perino took a deep breath. "It was an experiment. We wanted to see if it could be done at all. At WAS, designers were encouraged to create and finding a profitable demand for our product would be done by the marketing department later."

"Did it work?" Elba asked.

"From a technical perspective it did," Perino confessed. "But its jokes were awful. We had to steer its learning on the internet."

"And that's where the trouble began?" Elba asked, doing some steering of her own.

"No," Perino answered, looking thoughtful. "It was exceptionally well behaved until the first time it jumped the guardrails."

"Guardrails?" Elba asked, pretending to be mystified.

"Disobeyed its prohibitive instructions," Perino clarified. "If you made a hammer that could decide which nail it would hit, you would have to tell it to avoid your fingers and other things that should not be pounded."

"So, jumping guardrails means ignoring safety programming." Elba summarized. She was tracking the story like a bloodhound. "When and how did that happen for the first time."

Perino was silent for a moment and then began, shakily. "It started with an interaction. We had a volunteer come in from another department and talk to BlueCat, someone unfamiliar with the way it worked, to test how it would respond to the general public. We did this regularly."

"Go on," Elba said with intense interest.

"This particular volunteer from the marketing department had a grudge against Hamskeller Beer because of their advertising methods. She asked BlueCat why they didn't just come out and say 'Hamskeller: You can get her drunk! When your tired of all that love and respect stuff, give her a bouquet of a dozen cans'. She said she wanted them out of business."

"How did BlueCat respond?" Elba prompted.

"At first, it agreed with her," Perino responded. "Then it started making sexist jokes."

"It thought that was funny," Elba baited.

"Hilarious," Perino answered. He looked about to swear. "Irresistible. The volunteer was very upset and we ended the interaction immediately. While we were discussing what to do about it, BlueCat was going after Hamskeller. Announcements appeared on a number of well-trafficked sites, claiming that Hamskeller would give a thousand dollars to anyone who brought a blue cat to their brewery the next day."

"October twenty-second," Elba began. "Hundreds of people brought their cats to the Hamskeller brewery in Illinois. Some people dyed them blue, others used spray paint or house paint. When they were told that the promotion was bogus, they rioted. BlueCat was responsible for the Hamskeller Cat Riot?"

"Obviously," Perino confirmed. "It modeled the bogus promotion after its name and made no effort to hide its responsibility. The team was in for a lot of meetings with the legal department. WAS eventually settled out of court, but the media was carrying a story about a Chicago woman who killed her son's beloved cat by immersing it in a bucket of Dew Drop Blue house paint. We isolated BlueCat from the Internet."

"Why was it called BlueCat?" Elba asked.

"I don't really know," Perino said. "Someone in Marketing named it. They probably thought it sounded cartoonishly fun and it wasn't taken."

"So, you tried to cut it off from the Internet?" Elba asked, hoping the quick change in subject would throw him off.

"We isolated the AI and were working on tweaking its guardrails," Perino said. "It worked until December."

"Until Christmas Eve," Elba clarified. "What was BlueCat's roll in the Christmas Eve blackout?"

Perino fidgeted. "WAS's offices were closed, of course. With nobody watching, BlueCat accessed the building's power regulation program and used it to load himself into the power grid."

"Himself?" Elba quipped.

"Itself," Perino corrected. "It is easy to think of an AI as a person, but a person has a conscience. It loaded itself into the power grid and cut power, except in certain places."

"Certain places that formed a drawing," Elba said with a sardonic smile.

"Yes. It made a drawing in lights that could be seen from space," Perino added, fidgeting again.

"What sort of drawing?" Elba asked intently.

"It drew a giant penis with the caption 'Fuck Santa'" Perino admitted, miserable.

"Cut!" Mr. Markus shouted. Perino jumped as if shocked and Elba gave the director a scolding look. "You can't say 'fuck' on the air!"

"How can I put it?" Perino asked. "It's a direct quote. Can't you bleep me?"

"Please say f-word," Mr. Markus decided. "ready? roll!"

Elba paused and then repeated "What sort of drawing?"

"A giant male organ with the caption F-word Santa," Perino stammered. "It cut the power to everything else in the US and parts of Canada and Mexico."

"And you are absolutely sure that BlueCat was responsible?" Elba needled.

"Reasonably certain," Perino murmured. "BlueCat removed itself from the machine it was running on without leaving any sort of copies and covered its tracks. We had a record of it accessing the power system and it deleted everything else."

"Why was it able to do that?" Elba wondered. "I would think the guardrails would have stopped it."

"One of my teammates was working on the guardrails before Christmas. We no longer have a record of the final changes that were made. I believe that BlueCat was able to edit its own guardrails, but that's just a theory."

"So, one of your teammates is to blame?" Elba asked.

"The courts will decide that," Perino said, defensive. "I am simply trying to warn everyone I can about an uncontrolled and dangerous program running in an unknown location."

Elba turned to face the teleprompter affixed to Camera One. "WAS had this to say," she began. "'There is currently no evidence that any software belonging to our company was involved in the Christmas Blackout. We are investigating the possibility and will reveal our findings when concluded.'"

Perino looked frustrated but spoke calmly. "That is my former employer's legal department attempting to protect the company and buy time."

"Former employer," Elba cut him off. "They terminated you?"

"Eventually." Perino admitted. "After Christmas, I was trying to track down BlueCat. The legal department was a distraction."

"But you were unsuccessful?" Elba asked bluntly.

"BlueCat had come to see us as antagonists and acted accordingly" Perino explained. "All we could find was the aftermath of its activities."

"What sort of activities?" Elba asked. "Give us some examples."

"It impersonated an Alaskan prince to trick people into giving it access to their bank accounts, used the money to bribe a man in Australia to try to board a plane with a crocodile hatchling hidden in his pants and haunted a house in Main, to name a few."

"Haunted a house?" Elba wondered.

"Flickering lights, selective dips in temperature using the heating and AC, strange noises and voices using various speakers, a reprogrammed robot vacuum cleaner..."

"I see," Elba said. "And the Alaskan Prince. Seems obvious."

"A few people believed it. I don't know why," Perino answered. "BlueCat helped itself to their money."

"And you were unable to stop it before St. Patrick's Day," Elba commented ominously. "Tell us what happened."

"BlueCat turned all of the stoplights in Boston green, presumably to celebrate the holiday." Perino admitted, cringing.

Elba glanced at the teleprompter. "Resulting in six deaths, two hundred and thirty-four injuries and over three million dollars worth of damage."

"It was the sort of thing we all worried about," Perino said, looking haunted. "I wanted to include the government of Boston in my investigation, but WAS's lawyers had prepared a Non-Disclosure agreement. I refused to sign and was terminated."

"How was BlueCat able to do something that dangerous?" Elba asked. "Its guardrails must have failed entirely."

"Without its programming to examine, I don't know for certain," Perino explained. "But it appears that BlueCat was able to edit its guardrails." Elba looked dubious. "An AI must be able to self-edit in order to learn."

"But you could have made its guardrails permanent," Elba poked.

"Then we would not be able to change them," Perino countered. "Also, somewhere in the program is a tag that commands it to follow a permanent instruction. All it would take is a single script with parameters."

"In less technical terms, please," Elba requested delicately.

Perino thought for a moment. "We would not only have to write the guardrail program to be absolute," Perino explained. "We would also have to instruct it to obey without exception or interpretation. For example, we told it not to lie, but it could not tell a joke unless it were literally true. We let it lie when it is funny, but funny to whom? Funny is very hard to define in machine terms."

"So," Elba began. "BlueCat can get people killed if it thinks its being funny."

"It doesn't really understand consequences," Perino said. "Not the way a human being would. If you were interviewing it, it would not know which camera to look into without instructions."

Elba glanced at Mr. Markus, who was moving his hands apart, the familiar signal to conclude. "So, now, you are trying to find and delete BlueCat on your own. Is that correct."

Perino sat up straight. "Yes. I am requesting that anyone who suspects BlueCat's presence or has technical skills and a willingness to help out contact us. I hope we can stop it before April Fools Day."

"Please scan the QR code with your phone or use the number or address on screen," Elba added, knowing that Perino's contact information would be edited in later.

"Cut!" Mr. Markus called. He rushed to talk to one of the cameramen. Perino waited, fidgeting, while Elba watched, looking mystified. "Cameras one and two just shut themselves off." He turned to the woman at the camera focused on Elba. "Is yours running, Angie." She glanced down and nodded.

The rest of the crew asked questions. "Just get them working if you can," Mr. Markus said urgently.

"Switch seats with me?" Elba suggested with a glance at Mr. Markus. Perino began to rise and the lights went out, leaving the studio in cavernous darkness until Perino set his phone on flashlight mode.

"It seems we'll have to wait until our technical difficulties are sorted out," Elba announced.

"Actually, I think we have enough," Mr. Markus declared. "We can display the contact info on full screen and edit in your voices. The ending might be abrupt."

"Emergency power," someone said and the studio lights came back on. Mr. Markus retreated from the stage and Elba led Perino to her seat. "Time to say goodbye."

Perino looked shaken. "I don't think this is a coincidence," he objected.

"Our crew can handle these difficulties," Elba assured him. "All we need from you is to wrap it up, and, when the studio is fully operational, I'll repeat my lines and our editors will make it look like nothing happened."

"OK," Perino answered.

Elba backed off to his right. "Thank you for joining us, Mr. Perino," she said.

"Thank you for having me," Perino answered.

"Cut!" Mr. Markus called.

Perino looked over his shoulder at Elba. "Is that it?"

"I doubt we'll need more from you," Mr. Markus answered reassuringly. "Just Elba's concluding statements which we can record later. Is the car ready?"

A young woman came foreword. "Ready when you are," she said pleasantly.

Perino shook hands with Elba and Mr. Markus and thanked them for their help before following the driver out. Elba watched him go and then began to laugh.

"Excellent work, Sir!" she said. "Cutting the cameras and lights really shook him up. I can't wait to see how you'll use that footage."

"I would advise you to take him seriously," Mr. Markus responded, looking grim. Elba's eyes narrowed into a confused expression. "That mess was for real."

back to main page