As reported here and by my old boss Jeff Jarvis at BuzzMachine, the New York Times has finally decided to end its two-year experiment in subscription web content, TimesSelect.

As an avid reader of the Times columnists who were behind the pay wall and an occasional user of the Times archives (plus a crossword junkie), I was one of the 272,000 or so people who was willing to pay the $50 (£25) per year for access to the likes of Thomas Friedman and Paul Krugman. But even though I made the calculation that the content was worth paying for (or at least paying less for than a daily copy of the International Herald Tribune, where these columns and crosswords also run), the Times made the calculation that it was harming their business overall by limiting their distribution on popular search engines.

Jarvis writes:

TimesSelect’s brilliant cynicism was that, when forced to find something to put behind a pay wall, they came up with content that was, indeed, uniquely valuable — the columnists and archives. But this was also content for which there was no significant ad revenue at the time (advertisers buy ads in food and travel but not opinion sections; there is essentially no endemic advertising for blather). Thus they made the good college try to prove whether or not a pay news service could work without harming the ad revenue of the business. Even so, TimesSelect hurt the larger brand and its position in the marketplace, in the conversation, and in Google. It was a short-sighted strategy.

A short-sighted strategy, indeed. Mickey Kaus of Slate has been waging a crusade against TimesSelect for as long as the idea's been in the public domain. If you can read between the gloats below, he makes a point about the Times' arrogance that I find particularly salient given the business we're embarking on:

TimesSelect -- Pinch Sulzberger's attempt to put his prized columnists behind a subscription wall on the theory that they were so much better than free bloggers that people would pay for them -- is finally so doomed it's actually dead, dead, dead, as of midnight tomorrow.

But from my perspective, this decision reflects more than the dollars and cents that may have served as its basis. It's a reflection that the "communities" we refer to on the web really extend beyond the walls of any one site. The internet, itself, is the community, and the extent to which a piece of writing provokes a (reasoned) response anywhere is a pretty good measure of its worth. We may measure things in page views and unique users, but the interconnectedness of things makes it matter less, to a degree, where the response is posted. If you respond elsewhere to what I have to say, chances are that will pay a return of some sort by way of making my comments easier to find.

I've been having a few meetings with Big Media Companies recently in my role as a consultant (best gag of recent weeks: someone I met told a friend that they thought they were now officially a consultant, and the friend replied: "Neither am I"). One thing that's hit me very strongly is that all these companies have community elements to their sites. And actually none of them quite know why.

Some of them have a strategy, but it's at such a high level that the reality of managing the community on the ground is completely out of whack with what senior management think is happening. Some of them have provided community tools, but only out of some sense that this is the "right thing to do" for the 21st century media company. But none of them have sat down, looked into each other's eyes and asked the fundamental question "why do I need a community? What does this add to my business?".

I think there are several answers to the question "why". They differ by company, but they include:

• to extend the output of the company, for instance by allowing users to ask questions on message boards which are then answered live on air
• to deepen the engagement of some (or even most) users by encouraging them to discuss and participate
• to allow users to submit content which could be used by the company concerned in their media output
• to create relationships between the creators within the company and potentially domain-expert users, which can enrich the output of the organisation concerned.

There are almost certainly more, and these are only ingredients. Listing them is rather like saying a cake contains butter, flour and eggs. And of these four ingredients listed here, I think the second is the most dangerous, because it is vague and actually potentially misleading. Communities do deepen engagement, but only for a subset of users, and often quite a small subset. Conversely, the amount of management attention placed on them is out of all proportion to the impact they have on most users.

Also, what I've seen happen when companies are thinking about the second option is that they end up creating an environment where a subset of users can simply go into hyper-critical mode. It's like a shop opening an area where people can stand around slagging off their goods. Then the company hosting the critical community goes into hand-wringing mode, with some internally arguing that they have an obligation to allow the feedback, while others argue that the whole thing should be shut down.

What's happening, of course, is that the theoretical nirvana of community - engaged users who are enriching the output - is crashing up against lack of internal skills, lack of planning and in many cases lack of confidence. And by lack of confidence I don't just mean the confidence to allow users to criticise you on your own platform; I also mean the confidence to shut that community down when it starts becoming toxic. People talk about banning or suspending users like it's the nuclear option. It isn't. The nuclear option is shutting the thing down and rethinking why you did it in the first place.

So, in summary: user engagement with your output is almost always, and almost always by definition, a good thing. But if you don't think about why you want it, and thus about the best way to deliver it, you're headed for a world of pain. And never forget that it's your platform, even when it's peopled by your users, and you should deal with it confidently and in a way that benefits most users, not just the ones who have something to say.

Thinking of launching what we used to call an "online community" and what we now call a "sticky Web 2.0 social media experience"? Then read Alan Jones on moderating community and social media. In fact, read it, print it out and laminate it for use when talking to managers and lawyers who may not understand risk:

Will moderating content too much kill the pavlovian reward of "post-reply-reply-to-reply-repeat" that makes a social network sticky and compelling?

Why does that matter? Because woven tight into the good stuff, growing like weeds, you'll always find weird stuff also growing on your social networks, no matter what original purpose it has. Show me a user-generated content database, I'll show you some weird shit in there, every time.

Further, the line between "weird" and "bad" is wiggly, broad and fuzzy, with transparency set at >10% and sporting extra-aliased edges. Further, "bad" comes in a wide variety of flavours, including "bad for business", "bad for conservative families" and "bad for the legal budget line".

Alan comes down on the side of less moderation and more weirdness, which is pretty much what I'd expect from him, in the nicest possible way. And I think this story should be read alongside the slightly odd news that BBC News is making its "UGC hub" work "24/7", though interestingly they seem to be more interested in people as case studies rather than contributors.