The WSJ’s Permeable Pay Wall

I was reading a post from Jeff Jarvis on Rupert’s pay wall, when it occurred to me that not everyone knows all of the details of how it works.  It’s not as hard of a pay wall as the Times Select put up a few years back.  Both walls cover only some of the content, but there are a few situations where the WSJ lets anyone in.

The WSJ content is either flagged as Subscriber Content, which has a small key next to the headline, or is open to anyone.  However Subscriber Content is available to anyone if you are referred from one of the following sites:

• Google (news or search, anything Google.com, but not GMail)
• MySpace
• Digg
• Marketwatch
• Barrons (online.barrons.com)

I checked a host of other aggregators and search engines:  Bing, MSN, Yahoo, ShashDot, StumbleUpon, Twitter, Mixx, Newsvine, Facebook, Fark, Reddit, Drudge Report, and so on.  Nothing else hit.  MySpace, Marketwatch, and Barrons are easy to explain, they’re all News Corp. properties.  The New York Post, and other News Corp. properties, don’t get a pass.  Digg is probably the lone aggregator because it drives a lot of traffic, and was head-and-shoulders above similar sites back when they set the pay wall rules.  Those rules could probably use some revisiting.  Why Google but not Bing?  They don’t have a special relationship with Google, so I don’t see a reason to discriminate.

Occasionally you will see something labeled Subscriber Content which has been opened to everyone.  For example, I found a popular WSJ article on Twitter that is labelled Subscriber Content which can be read in full even if you aren’t a subscriber.  How that happens I’m not sure.  Perhaps if a subscriber shares a link for some protected content, they might decide to make it free, either automatically or by alerting a human.  The Drudge Report frequently links to WSJ articles, but surveying a few days of links I couldn’t find examples of Subscriber Content being linked to.  Although I would wager that if Drudge did link to Subscriber Content, they’d open it up.

So the WSJ has a permeable pay wall.  Jeff’s point about not getting as much Googlejuice is a good one, but I doubt the WSJ is losing out on much.  They’ve probably had a high rank from the day Google launched, and enough links around the web that their rank won’t be falling anytime soon.  They certainly loose out on some traffic by not giving bloggers and newer social networking sites the same pass as Digg and MySpace.

How much are they losing?  Who knows.  Looking at referer data from compete.com and alexa.com, they don’t look too different from their peers.  Their traffic has been steadily increasing.  In fact, they’ve done a better job at adding pageviews than the NYTimes, which is down based on Alexa stats, and flat based on compete.com.

What does that tell me?  There are other important drivers of web traffic than just whether you have a pay wall.  Hmm, maybe it has something to do with quality, and Rupert is on to something.

Please, Someone Sue Google News in an American Court

There is an interesting article in the New York Times today that discusses excerpting by online news sites.  I’ll avoid quoting it heavily, if you want to know what it says, follow the link.

Two of the companies mentioned, All Headline News and the Huffington Post, are giving the industry a bad name.  The former is being sued by the Associated Press.  They claim that AHN does no original reporting and merely rewrites content provided by the AP.  Perhaps in response, AHN is now looking for citizen journalists nationwide.  It all seems questionable to me.  How do we, or indeed AHN, know that these citizen journalists are doing original reporting, and not rewriting other’s content?  What exactly are the credentials of AHN and to what standards are their contributors held?  Will they use citizen journalism as a license to steal, a job where a poor-mans version of Jayson Blair would thrive instead of being censured?  As for the Huffington Post, I’ve already said their excerpting policy seems excessive.

Both of these companies seem parasitic, although the Huffington Post otherwise offers good blogs.   I suppose its up to the content owners to determine whether whatever utility they provide is worth the blood they suck.  It worries me, because Daylife and plenty of other useful companies could get caught in the crossfire.  Not every symbiont is a parasite, so don’t kill all of the insects, just the mosquitoes.

This is one reason I deeply regret that the GateHouse Media vs. New York Times Corp  case didn’t go to court.  Building up good case law requires good cases, and All Headline News is not a good case, nor would the Huffington Post be one.  Nor would the defendents in either case have sufficient funds to retain a top notch legal team, whereas The New York Times Corp did.  The only other deep pocket that might provide a good case?  Google News.  Please, someone sue them in an American court over excerpting.

Don’t take me wrong, I like Google News, and I don’t want them to loose.  I want good case law.

New York Times Article API Offering Faceting Search

The New York Times released an article search API today, and not just any article search API, but an excellent one.  You can filter results by a large number of facets.  For example, articles about green energy from the business section that mention Google:

http://api.nytimes.com/svc/search/v1/article?query=green%20energy%20nytd_section_facet:[Business]%20org_facet:[GOOGLE%20INC]&api-key=[your-api-key-here]

You can also request facets and counts for the search query:

http://api.nytimes.com/svc/search/v1/article?query=mortgage%20crisis&api-key=[your-api-key-here]&facets=des_facet,per_facet,org_facet

The API is in fact more capable than their site search, which although in the advanced mode will show facet counts at the bottom, is not as flexible in allowing you to progressively filter by multiple facets, or restrict the types of facets returned.

There are some gaps.  The facets are sometimes ambiguous, for example if you search for “George Bush”, you will get a facet for “George W. Bush”, and one for “George Bush”, with about equal counts, so although the vocabulary is controlled, it is not perfect.  There are several API providers who would not let an ambiguous facet like “George Bush” through.  There are also no provisions for associating a person or organization facet with any external sources like DBPedia, which Calais and Zemanta both do.

They’ve been cranking these out once every month or two, but this is the first one that opens up their own content.

New York Times Considering a Return to Access Fees

The New York Times is seriously considering charging for online access.  Two years ago some of their content was behind a pay wall, and the Wall Street Journal still reserves much of their content for subscribers.  There were rumers a year or two ago that WSJ would drop the pay barrier, but it never happened.  As I said earlier, it was a smart move.  It would also be wise for the NYTimes to start charging.  

There are only three good options for news organizations as I see it:

1. Keep their current business model, ad-supported free access online, with or without a print edition, and cut back significantly on your news room.  They’ll have good coverage for a few niches, and link to other organizations.
2. Charge for online access and hope that you’re left standing after many of your competitors go bankrupt or cut their news rooms to the point where you have enough of an edge in quality to support  the access fee.
3. Hope that enough competitors will die off that you can collect the web traffic and ad revenue needed to support a large news room without an access fee.

The news industry as I said earlier is going through a shakeout with the shift to digital.  It’s not the only possible future, but I see a few large news outlets that charge access fees and have respectible news rooms, and a lot of smaller outlets with a skeleton crew and an ad-supported site.  Readers have a real demand for good news, and it will be satisfied one way or another.  

Option 2 will eventually be realized by a few organizations.  You can waive your hands about a mass of niches, that organizations will focus on a few small areas and everyone will link to eachother, and that having everyone take option 1 is viable.  This denies the benefits of branding and any economy of scale.  Collecting good writers under a single masthead has genuine benefits, and a writer for the NYTimes or the WSJ, even though I may not be familiar with them, is more trusted than a lone blogger or a small outlet.  They have experienced editors that protect their reputation.  This is the economy of scale and the benefit of branding.  

So I argue that when the industry has reached equilibrium after this disruption to their business model, the industry will have resized by eliminating a significant number of outlets, and that a handfull will have significant news rooms and offer quality comparable to what is now available.  But, like too many rabbits fighting over a small patch of grass, they are all going to be starved for a while, until the weak ones die off.  Everyone should be thinking about how to make it past the die-off, and whether they want to try for option 1 or option 2.  Trying for option 3 would be gruesome, and you’d need a large war chest.  I also think that it does not sufficiently capitalize on the demand for good news, and that a company finding itself in that situation would be able to extract more revenue by instituting a fee, so ultimately we’re back to option 2.    

The New York Times is in a good position to survive while still maintaining a sizeable news room, although not as good a position as the Wall Street Journal.  With respect to their consideration of an access fee, it is perhaps a dubious proposition now since good free alternatives still exist, but the proposition will only get better as those alternatives fold or decline in quality.

That’s my quick analysis.  Perhaps some day I’ll crank through a few numbers. Bottom line, if they want to survive in something resembling their current form, which I ernestly hope they do, a NYTimes access fee is a question of when not whether.  Precisely when depends on financial details to which I don’t have access.