October 12, 2009

Computers may be good at crunching numbers, but can they crunch feelings? Yes! A new kind of search scours the Internet for users' emotions and translates them into hard data.

The rise of blogs and social networks has fueled a bull market in personal opinion: reviews, ratings, recommendations and other forms of online expression. For computer scientists, this fast-growing mountain of data is opening a tantalizing window onto the collective consciousness of Internet users.

An emerging field known as sentiment analysis is taking shape around one of the computer world's unexplored frontiers: translating the vagaries of human emotion into hard data. This is more than just an interesting programming exercise. For many businesses, online opinion has turned into a kind of virtual currency that can make or break a product in the marketplace.

Yet many companies struggle to make sense of the caterwaul of complaints and compliments that swirl around their products online. As sentiment analysis tools begin to take shape, they could not only help businesses improve their bottom lines, but also eventually transform the experience of searching for information online.

Several new sentiment analysis companies are trying to tap into the growing business interest in what is being said online. Social media used to be this cute project for 25-year-old consultants, but now, top executives are recognizing it as an incredibly rich vein of market intelligence.

A subscription service was introduced, allowing customers to monitor blogs, news articles, online forums and social networking sites for trends in opinions about products, services or topics in the news.

Translating the slippery stuff of human language into binary values will always be an imperfect science, however. Sentiments are very different from conventional facts. The simplest algorithms work by scanning keywords to categorize a statement as positive or negative, based on a simple binary analysis ("love" is good, "hate" is bad). But that approach fails to capture the subtleties that bring human language to life: irony, sarcasm, slang and other idiomatic expressions. Reliable sentiment analysis requires parsing many linguistic shades of gray.

As sentiment analysis algorithms grow more sophisticated, they should begin to yield more accurate results that may eventually point the way to more sophisticated filtering mechanisms. They could become a part of everyday Web use. As sentiment analysis is becoming a standard feature of search engines, such algorithms could begin to influence both general-purpose Web searching and more specialized searches in areas like e-commerce, travel reservations and movie reviews.

As search engines begin to incorporate more and more opinion data into their results, the distinction between fact and opinion may start blurring to the point where facts all come with points of view.

RELATED

Internet