James Hilston, Post-Gazette Graphic: How Peekaboom works and why you should play. Click image to view graphic.

With a few keystrokes on his laptop computer, Luis von Ahn calls up the Web site for something called the ESP Game and finds the information he wanted: 37 players are logged onto the site, playing his game.

"One guy has played 100 times in the last 24 hours," he noted with not a little satisfaction.

Most people might say that player is frittering away his time online. Even the player likely would admit as much. But whatever brain power the player is devoting to the game will not go wasted; von Ahn will put it to good use.

The ESP Game is no ordinary online game, but a clever way of using the Web to painlessly harness the brainpower of computer users. And von Ahn, who will soon complete his doctorate in computer science at Carnegie Mellon University, is ready with yet another game, this one called Peekaboom.

While the ESP Game was designed to generate descriptive labels for photographs and other images, Peekaboom is intended to help teach computers to see.

Computer vision is still at a primitive stage, in part because it takes so much time to teach computers how to identify objects -- a process that involves highlighting objects or features in images for the computer.

"There aren't many humans who are willing to sit down and teach a computer to see," said Manuel Blum, a theoretical computer scientist at Carnegie Mellon University. But as von Ahn has demonstrated with the ESP Game, there are plenty of people who will do so if they think it's fun.

"People don't do it because they want to help," von Ahn said. "They just do it because they want to pass the time."

It's much the same approach that Tom Sawyer used to trick his friends into whitewashing a fence for him. And thus far, it's been just as successful.

Since the Post-Gazette first wrote about the ESP Game in October 2003, more than 80,000 people have played the game and in the process have generated more than 10 million descriptive words for 1 million images.

Those labels could be used as keywords by Internet search engines to search for those images. Blum and von Ahn said they are in talks now with at least one company regarding possible use of the game or the labels.

Playing the game

In the meantime, the labels and photos will be used as the basis for Peekaboom, a two-player online game. It is located at www.peekaboom.org.

In the game, which graduate student Roy Liu has been programming since September, one player is given an image and a word that describes the image or one element of the image. The object of the game is for the first player, designated as Boom, to get the other player, designated as Peek, to correctly guess the word.

Peek initially sees only a blank screen, but as play proceeds Boom reveals the image, little by little, by moving his cursor over the image.

In some cases, simply revealing the element of the image described by the key word isn't a sufficient clue. The trunk of an elephant is one example. "If you just show the trunk, it doesn't look like anything," Blum said. So Boom might reveal most or all of the elephant; he can then highlight the trunk, using a feature of the game called "pinging."

The images are the same ones used by the ESP Game and the descriptive words are those generated by ESP players.

When the word is correctly guessed, or when the players agree to pass on an image, the players switch roles and play resumes. The players receive scores based on the number of correct guesses they can make in four minutes.

"When you do well, you feel smart," said Josh Falk, a 10th-grade student from Point Breeze, who tried Peekaboom while participating in a Carnegie Mellon University summer program for local high school students called Andrew's Leap. "And when you do poorly, you think the other person is stupid, which also makes you feel smart."

Players are paired anonymously after they log in to the game.

The process of revealing objects, or highlighting images within the larger context of the photo, is the sort of thing that researchers in computer vision must do to teach computers to see.

Image problems

In the early days of machine vision research, it was assumed that computers could learn to identify an object, such as a car, or a spoon, or a face, if it was given the rough geometry of the object, said Alexei Efros, a computer vision researcher at Carnegie Mellon. But that wasn't a successful approach.

A better approach to teaching a computer how to identify an object, such as a car, is to show it lots of images of cars, of various makes and colors, taken from a variety of angles and distances and under a variety of lighting conditions, he said.

But obtaining "segmented" images -- images that have the desired object highlighted -- is laborious. One of Efros' projects uses 300 such images. Researchers at the Massachusetts Institute of Technology have been collecting a database of images that now number in the thousands. "That's huge," Efros added.

Peekaboom might eliminate that bottleneck, however.

"What Luis has been telling me is that he can give me on the order of millions" of images with labeled objects, Efros said. "This is a huge jump in quantity."

It remains to be seen what the quality of those images and labels might be, but experience with the ESP Game suggests that the quality won't be half bad, he said.

"This really has the potential to solve the most painful problem in vision research right now," he added.

And Peekaboom isn't the end of von Ahn's plans. Though he won't discuss what the next games might be yet, he knows what he wants them to accomplish. The next one will generate explanatory sentences for images -- something that might be useful for Web searches by blind people.

And he envisions another game that could be used to teach computers about "things that everybody knows," but that computers are clueless about -- how to open a door with a doorknob, what happens when you push a floor button on an elevator, etc.

In the meantime, he and Liu will be troubleshooting and improving Peekaboom, which has been operational for about a month now.

"Sometimes, the title doesn't match the picture well," said Paul Rubritz, an Andrew's Leap participant from Marshall. And Andrew Drake of Plum complained that the selection of photos sometimes seems limited; "I saw the same logo in five consecutive games," he said.

"There are glitches," Blum acknowledged. "It is a beta version. But there's no way to fix it until someone uses it."

	James Hilston, Post-Gazette Link: The Peekaboom Web site. Click photo for larger image.