Program helps boost computer sketch recognition

By Patrick Anderson
PBN Staff Writer

Brown University assistant professor James Hays knew computers have a good eye for art, especially detecting texture and shading in life-like representations of real-world images. More

To continue reading this article, please do one of the following.



TECHNOLOGY

Program helps boost computer sketch recognition

PBN PHOTO/NATALJA KENT
DRAWING BOARD: James Hays, assistant professor of computer science at Brown University, demonstrates different aspects of his sketch-based recognition program.

By Patrick Anderson
PBN Staff Writer

Posted 2/4/13

Brown University assistant professor James Hays knew computers have a good eye for art, especially detecting texture and shading in life-like representations of real-world images.

But Hays and fellow researchers wanted to find out how artificial intelligence reacted to more rudimentary art, the kind of sketches amateurs might dash off on a napkin or notebook.

Could a computer identify the abstract shapes and outlines commonly used across cultures to represent sharks, cars or zebras? Could a computer play the party-game “Pictionary”?

To their surprise, Hays and two visiting researchers from Berlin found they could teach a computer to recognize our cruder visual renderings, if not quite as well as other humans can.

Thanks to the emerging capabilities of crowdsourcing, Hays and colleagues created a program that correctly identified the subject of a simple sketch more than half the time. A random selection would have gotten the identification right less than 1 percent of the time and humans generally get it right three-quarters of the time.

“I was positively surprised,” Hays said. “Part of the reason that there are a lot of recognition problems in computer vision is that the part they are good at is texture. A computer is worse at things defined by shape, with no texture and just outline.”

And simple, abstract outlines are the most common ways people, especially those without great artistic talent, render images when trying to communicate through drawings.

So the capacity for computers to recognize simple, iconographic human drawings could have vast uses, such as pictorial Internet searching.

“Sketch-based search is an existing field, but now that we don’t just rely on matching sketches and photographs, it might be better,” Hays said, “allowing children, maybe people who are not literate or don’t speak the right language, to interact with a computer.”

Hays’ background is in computer graphics and vision and it was only in discussions with the visiting researchers from the Technical University in Berlin, Mathias Eitz and Marc Alexa, that the three began thinking about sketch recognition.

Software already existed that allowed computers to match images, such as photographs or professional police drawings, but no one had tried to teach a computer to recognize hand-drawn visual symbols, such as four legs and stripes representing a zebra, on a conceptual level.

Next Page
No comments on this story | Add your comment
Please log in or register to add your comment
Calendar
PBN Hosted
Events

Register today for the First Annual PBN Manufacturing Awards Dinner and Ceremony, Thursday, April 24th, at the Bryant University Bello Center. Click here for more information, including the list of winners.
  • Business Women
    Accomplished. Confident. Wicked Smart. Nominate women you know who deserve speci ...
  • 40 Under Forty
    It's the 10th Anniversary of 40 Under Forty, and PBN is planning a special event ...
  • Healthiest Employers
    Enrollment now open for the 2014 Healthiest Employers Program. Please take a mom ...
Advertisement
Purchase Data
Book of Lists
Lists
Book of Lists cover
PBN's annual Book of Lists has been an essential resource for the local business community for almost 30 years. The Book of Lists features a wealth of company rankings from a variety of fields and industries, including banking, health care, real estate, law, hospitality, education, not-for-profits, technology and many more.
Data icons
Data can be purchased as single lists, in either Excel or PDF format; the entire database of the published book, in Excel format; or a printed copy of the Book of Lists.
  • Purchase an e-File of a single list
  •  
  • Purchase an e-File of the entire Book of Lists database
  •  
  • Purchase a printed copy of the Book of Lists
  •  
    National
    Local
    Latest News