MIT instructor Philip Greenspun has an excellent
review of Internet telephone service.
Greenspun's review helps illustrate the sorry state of newspaper product
reviews. Take, for example, Walter Mossberg of the Wall Street Journal, who
the New Yorker makes close to $500,000 per year. Mossberg is easily the
dean of technology reviewers, and has written some helpful reviews,
including one that rightfully sent many customers to MusicMatch, an underdog
maker of MP3 players. But his review of Internet
telephone provider Vonage is a letdown.
Greenspun notes that Vonage is among the most expensive Internet
telephone providers. Mossberg doesn't. Greenspun notes that Vonage has
limited service in Europe. Mossberg doesn't. Greenspun also tries to use the
Vonage as his primary telephone number and uncovers all sorts of customer
service problems in the process. For some reason, maybe because his review
was written 10 months earlier, Mossberg doesn't.
I hate to pick on Mossberg, who seems usually to work harder and know
more technically than other reviewers. There is a broader problem with
newspaper product reviews. The problem is the process.
For one, greedy publishers set insane deadlines, so the reporter reviews
the product in an absurdly artificial span of time. For example, I know a
video game reviewer who had one weekend to play two or three different
titles. The review was due Monday. (Of course, this is for illustrative
purposes only, I'm sure *her* review won't suck! Especially if she's
Look at Greenspun's review. He spent 10 months wrestling with Internet
telephone service. Back in 1999, I wrote an online
review of an Internet Service Provider after six months. Newspaper
editors simply will not give this kind of time.
Newspaper product reviews also suck because the reporter isn't a tough
customer. He's an artificial customer, one who half the time hasn't paid for
the product in question. And the writer doesn't review the product because
he honestly wants it. He reviews it because he's the guy who does the
reviews. Or because he got three of them for free in the mail. Or because
his editor had an idea about how cellphone mp3-playing videogame handhelds
are the wave of the future.
I recently bought some outlining software for my PalmOne handheld comuter
called ShadowPlan. This little bitty piece of software has its own Yahoo
discussion group with over 20,000 messages! These people spend weeks
arguing passionately over the optimal way to implement David Allen's Getting Things
Done in this little outliner. They complain about default folder
locations, Save dialog boxes, syncing, the desktop software, how files are
launched, importing files, linking the outliner to the other Palm software
-- the list is nearly endless.
Newspaper reviewers, in contrast, miss the little details that are so
critical to the effectiveness of a product. I'm not going to say Mossberg
was wrong to recommend the
iPod. But it took a regular user of the product like Dave Winer to point
out that it's a difficult platform for audio books and unintuitive when
you hook it up to your car.
The biggest tipoff that newspaper reviews suck is that they so often
sound like promotional material for the service or product in question. For
example, in Mossberg's Internet telephone review, he spends just half a
sentence trying to figure out why anyone would switch to Internet
telephones. Maybe because they hate the phone company? Bzzzt, wrong. If a
phone user is not buying Internet access from the phone company, they are
buying it from the cable company. And if they hate the phone company, they
*really* hate the cable company.
Anyway, after the half a sentence thinking about people, Mossberg starts
with the product pitch. Saves you money! Unprecedented flexibility! Easy
setup! Call anywhere in the world! Three plans available! They'll throw in a
free gift! The kind of stuff you'd see on a brochure, press release or
Then Mossberg gets into the product flaws, something you won't see in any
sales literature. And I think that's great, and it's what he gets paid the
big bucks for, and he doesn't do a criminally bad job of it.
But Greenspun's whole approach is different. He's relentlessly focused on
what an Acutal Person Actually Using the Product would want.
He gives some great reasons to switch to Internet phones -- cheap
international calling, cheap or free long distance, save some money, one
phone number for multiple phones, and so on. His pitch, actually, is more
effective than what you would get on the Vonage
home page, probably because he is thinking about customer needs rather
than how to make lots of money.
And then Greenspun gets into the nitty gritty of actually using the
product -- transferring your number, making those promised international
calls, getting the thing to forward to your telephone. Does it help that
Greenspun actually wants to make this service work so he can keep using it,
that he's not just trying to write up a quick review and hit another