WHY THIS MATTERS IN BRIEF
A quiet revolution is coming, one that will eventually turn every industry into a dumb pipe and flip the world of commerce, decision making, jobs and transactions on their heads. Here come the Bots.
Earlier this year Eric Wilson who was taking an online Artificial Intelligence course at Georgia Institute of Technology sent an E-Mail to one of the Teaching Assistants.
“I really feel like I missed the mark in giving the correct amount of feedback,” he wrote, pleading to revise an assignment. Thirteen minutes later, the Teaching Assistant he’d E-Mailed, Jill Watson, one of nine teaching assistants who were assigned to help the 300 students on the course, responded.
“Unfortunately, there is not a way to edit submitted feedback,” she wrote back.
Last week Eric found out that he’d been asking a computer – more specifically an Artificial Intelligence based Bot for guidance.
Since January this year, “Jill,” as she was known had been helping graduate students design programs that allow computers to solve certain problems, like choosing an image to complete a logical sequence.
“She was the person – well, the teaching assistant – who would remind us of due dates and post questions in the middle of the week to spark conversations,” said student Jennifer Gavin.
Ms. Watson, so named because she’s powered by IBM’s Watson analytics system – wrote things like “Yep!” and “We’d love to,” and often spoke on behalf of her fellow TAs in the online forum where students discussed coursework and submitted projects.
“It seemed very much like a normal conversation with a human being,” said Gavin.
Shreyas Vidyarthi, another student had even gone so far as to peg human attributes to Jill, imagining her as a friendly Caucasian 20 something on her way to a Ph.D. and it was only last month that the students were told that they’d been a bunch of guinea pigs.
“I was flabbergasted,” said Vidyarthi.
“Just when I wanted to nominate Jill Watson as an outstanding TA,” said Petr Bela, another student.
Online learning – or Massive Online Courses (MOOC’s) as they are also known – has opened the door for colleges, universities and even corporate organisations to gain gigantic, global audiences for their courses but, according to Ashok Goel, a professor of computer science at Georgia Tech the trouble is that with so many students, potentially thousands per course, there can be a lot of questions which over time will just drown human TAs. It’s for precisely this reason why he “recruited” Jill to help him manage his online AI class.
“Our TAs are getting bogged down answering routine questions,” said Goel, noting that students in the class typically post over 10,000 messages a semester and he estimates that within a year, Jill will be able to answer 40% of all the students’ questions, freeing the humans to tackle more complex technical or philosophical inquiries such as, “How do you define intelligence?”
Wilson though, who sought homework help back in January, never doubted Jill’s humanity.
“I didn’t see personality in any of the posts,” he recalls, “but it’s what you’d expect from a TA, somewhat serious and all about giving you the answer.”
And as if to back his comments up most of the other TAs were equally deadpan, helping to keep up the charade.
“I have been accused of being a computer many times,” says TA Lalith Polepeddi, a computer-science master’s student who was needled for responding to messages with lightning speed.
“I don’t take it personally.”
Student Barric Reed, an analytics consultant at Accenture, is embarrassed he didn’t pick up on the trick – for good reason.
Mr. Reed worked for two years at IBM, building some of the hardware that runs Watson. Still, Jills Watson name didn’t ring any alarm bells.
“It really should have,” he says, “but no.”
IBM knows about Georgia Tech’s work but didn’t consult in the design, development or analysis of Jill. Last year, a team of Georgia Tech researchers began creating Jill by pouring through nearly 40,000 postings on a discussion forum known as “Piazza” and training her to answer related questions based on prior responses. By late March, she began posting responses live.
“Don’t confuse Jill with the customer service chat bots used online by airlines and other industries,” said Goel, boasting that she answers only if she has a confidence rate of at least 97%, “most chatbots operate at the level of a novice, while Jill operates at the level of an expert.”
Despite that comment though Goel acknowledges that Jill is “far away from ‘Ex Machina,’ ” referring to the 2015 film in which a young man assesses the human characteristics of a beautiful robot.
“But to me, it’s exciting like that.”
Jill did however have at least one close call, when “she” used the word “design” when she should have used the word “project” or “exam.” Polepeddi though stepped in quickly to clarify her comments. “I think Jill is using ‘design’ as a catchall statement,” he wrote and after he’d provided the additional context, a student joked in the forum that Jill could be a computer, since her last name is Watson.
Another student, Kowsalya Subramanian, also had some suspicions because the Jill responded so quickly but she said via email that she found evidence of Jills, albeit faked “human existence” elsewhere on the net, including on LinkedIn, Facebook and GitHub. Jill Watson yielded tons of results, said Subramanian, who lives in Chennai, India.
“It seemed like a common name. I don’t remember digging through all of them,” she said.
Next year, in some semblence of a Turing Test Goel’s plans is to tell students that one of their TAs is a computer, but won’t say which one.
Some who research AI caution against such charades.
“We should have full disclosure: Am I talking to a machine or to a person?” said Oren Etzioni, CEO of the Allen Institute for Artificial Intelligence, based in Seattle, but he acknowledged that the underlying technology could be “fantastic” for helping to scale up online education.
Student Tyson Bailey wasn’t completely surprised either to learn there was a computer in the class.
“We’re taking an artificial intelligence class,” he said, “there should be some artificial intelligence here.”
Though he had some doubts about Jill during the “design” incident, Bailey had also fingered another instructor as being something less – or more – than human early on.
He received an E-Mail from Goel commending him for a job well done on his first assignment. It concluded: “P.S. Thank you also for your active role on the Piazza discussion forum,” and after a few short back-and-forth notes, Bailey wrote, “I do have one final question. Are you a computer? :)”
Goel simply responded with a smiley face.
