The President of the United States and your next video rental might both
someday be chosen over the Internet through what's called collaborative
filtering. This emerging technology came up over lunch recently at my town
house in Boston. The guest of honor was MIT's Tim Berners-Lee, father of the
World Wide Web and now director of the Web Consortium at __http://www.w3.org_,
where collaborative filtering is much discussed. Berners-Lee talked about
whether, after Netscape, there are any entrepreneurial opportunities left on
the Web. We agreed that out there on the Web there's still a promising mix of
Route 128, Silicon Valley, and Dodge City.
One gunslinger joining us for lunch was Yezdi Lashkari, advanced technologist
at Agents Inc., a recent MIT Media Lab spin-off in Cambridge, Mass.
Lashkari's Web site, called Firefly, is a place to get music recommendations,
at _http://www.ffly.com_. Firefly uses collaborative filtering, which Lashkari
was happy to explain. Collaborative filtering is made up of a variety of
technologies for weighing ratings from an on-line community and transforming
them into recommendations. I am now more convinced than ever that
collaborative filtering will evolve into one of the big technologies of the
Information Age.
My favorite example of collaborative filtering is not presidential elections
but video rentals. It goes something like this: You are returning a video you
watched last night. The rental store clerk asks you to rate the video from 1
to 10. He types your rating into the store's computer and hands you your next
video. You take it home and love it. Or the clerk offers you a selection of
videos that you have not seen yet, that he has in stock, and that you,
depending on your mood, are very likely to enjoy. Or all this happens in front
of your interactive television via video on demand.
Collaborative filtering through the Internet would not be just for videos but
also for books, restaurants, cars, resorts, blind dates, places to go with
blind dates, newspaper columns, operating systems, Web pages, software
components, presidential candidates, and pizza.
Imagine a database, a sparse million-by-a-million matrix with each row a
video rental customer and each column a video. Each element of the matrix is
the rating, if any, of that video by that customer. And now the question is,
which videos would the database recommend that you rent?
Using a collaborative filtering algorithm, a video recommendation server
would work its way through its database looking for other customers who have
rented some of the same videos you have rented and whose ratings tend to
correlate with yours. Then, for each video that you have not rented, the
filtering computer asks how that video fared among customers with tastes
similar to yours. The more ratings the database has, the more likely you will
be happy with its recommendations, or so the theory goes.
When a collaborative filtering system asks you to rate one of its items,
there are two reasons for you to do so. The first is to help others find items
appropriate for them. The second is to help yourself -- your ratings identify
which community members are the best judges of what you might like.
Personal newspapers might be assembled through collaborative filtering.
Instead of just specifying keywords of interest, you rate news stories you
have read. As news stories appear, they are at first recommended using
keywords, but, as ratings accumulate, you are then offered stories rated
highly by others who tend to prefer the same stories you do.
So how might we use collaborative filtering to someday elect a president? Set
up a system on the then-universal Internet for collecting citizen ratings of
alternative government policies. Ask each candidate to rate these policies,
too. Ask citizens to rate the candidates. In the end you vote for candidates
that most other people with similar convictions would vote for too. This would
be disintermediated democracy, if you can stand it.
We'll inevitably get to such a system, I'm sure, only more complicated, so as
to preserve jobs for lobbyists, who, after all, are entitled to lunch, too.
And if not lunch at my house in Boston, they can try collaborative filtering
through The Boston Restaurant Guide at _http://www.hubnet.com_.