“Webmasters who do not wish their sites to be indexed can and do use the following two lines to deny permission: User-agent: *, Disallow: /,” wrote Google News manager Josh Cohen, referring to the well-known robots.txt method of excluding search crawlers… It politely reminds publishers that, if they really want to block Google, they can do so using an old fashioned protocol rather than tinkering with intellectual property law.
Being a longtime Navy guy, I can also appreciate the reference later in this article to telling publishers to “RTFM.”
Bottom line is that publishers understandably might like to reformat the news and information environment to their liking and financial benefit, but they can’t realistically claim to have no alternatives. It’s more a matter that they don’t like the alternatives and want better ones. Good luck with that.