The Society for New Communications has issued an executive summary of a forthcoming study, the "ROI of Press Releases."   The study results state the obvious -- the terrain and tactics of press release distribution have changed.  But some of the legacy requirements remain.

PR professionals stick with their traditional goals of announcing news and increasing thought leadership.  Marketers view SEO and reaching consumers as the most important tactics in this case.  Small business owners position online press releases as business development tools and potential profit centres. 

What struck me most about the survey results was the lack of a clear winner in criteria for measuring the success of online press releases… regardless of your profession. As seen by the clustering of the survey responses (multiple selections allowed), those who rely on PR remain unclear about the best metrics: 

• Online press release republished on websites (79.6%)
• # times online press release viewed online (76.8%)
• Articles generated from the online press release (75.4%)
• Media interview requests (74.2%)
• Traffic to organization’s website (~73%)

What do these results say to me?  First, more must be done to prove the value of an online PR function.  Second, traditional press release success criteria, such as distribution and exposure, remain the default metrics for PR success, whether online or offline.  What this study fails to engage with (and the study authors note this limitation) is that we need to move to more robust models of capturing larger indicators of success, such as product sales from online articles and related behavioural shifts that result from online and offline PR).    

Granted, it is early days in developing metrics for an emerging media relations tool.  However, PR metrics must not only measure reaching the media but reaching the public directly.  Perhaps charities, such as one mentioned in a recent Metrica blog post, hold the key to proposing commercial angle metrics structure to how best to measure the success of our evaluation criteria.

Last week, while at the PRSA International conference, Richard posted on the Christian Science Monitor's decision to publish online only from April 2009.  This is an exciting move within the media industry as there is an increasing amount of evidence that audiences are looking to the internet for their daily news fix.  Paul Hender will be sharing some of that data on Metrica's Measurement Matters in the next couple of days so watch this space.

From an evaluation perspective, Metrica has been helping its clients to adapt to this shift for some time now.  We have been working with them as their PR plans have incorporated online media outlets and social media.  

We would love to know your views on the subject.  Let us know what you think about the CSM's decision.  Is it the beginning of an inevitable trend which will spell the end of blackened fingers, as the good ol' fashioned paper gets phased out?  Or will the big national papers always retain a hard copy?  Does the same hold true for regional papers?  Or will declining ad revenue force the smaller regional publications to rein in costs by going online only? And will the proliferation of web technology and online interaction change the entire way which we find our news?

Place your votes below and please feel free to add a comment explaining why.  We will post the results at the end of the month.

“We have the luxury — the opportunity — of making a leap that most newspapers will have to make in the next five years" explained John Yemma, the Christian Science Monitor's editor yesterday while announcing that the paper would end its daily print edition in April 2009. 

At this week's PRSA International conference, the paper was described to me as 'one of the world's best newspapers that nobody reads'.  This could be because of the unique way that it is structured - as the NY Times explains in an excellent article:

"an anomaly in journalism, a nonprofit financed by a church and delivered through the mail. But with seven Pulitzer Prizes and a reputation for thoughtful writing and strong international coverage, it long maintained an outsize influence in the publishing world."

This is the first of the media's big brands to move towards this strategy, and it should be applauded.  With the advent of the internet, blogging platforms and social media, audiences are fragmenting into smaller and smaller special interest groups, creating and consuming only the content that they want.  They want this information in real time. 

This of course is a great advantage of the online strategy.  No more do audiences waking up on the east coast of America turn to printed newspapers only to find that the match report that they wanted to read from LA missed the print deadline.  It's now there in all its glory in the online edition, report, photos, and in all likelihood some video highlights too.  And if it's not in their version of the paper, you can rest assured that one of their friends on Twitter will have told them the score and linked to the best resources for all of the details before the cornflakes are even out of the cupboard.

If your media monitoring and media analysis / evaluation is not currently covering the new electronic and social media, this is the wake up call that it's time for you to reconsider.  The media world is changing, and it's vital that we all change with it.

Finally, if you're keen to know more about Twitter please feel free to connect with me - you'll find me on the site with the username @richardbagnall. I'll look forward to meeting you there.

At the PRSA Point of Connection Conference yesterday I attended a joint presentation by Katie Paine of KDPaine and Partners and Shonali Burke (twitter: @shonali), until recently the head of Comms at the ASPCA. 

Shonali shared with us some media analysis data from her tenure with the ASPCA with which she was able to prove the value that the PR function brought to the not for profit.  By aligning the PR objectives to the organisation's objectives, focusing on them and instigating a thorough and credible PR measurement programme, Shonali was able to prove the value of the communications function.  The ASPCA's metrics didn't stop at basic media evaluation of outputs, but also went on to disaggregate and correlate these findings against revenue gained through donations.  The result of all of this was that she was able to justify a significant increase in resource and head count for her team.

As I sit here this morning watching yet more dreadful financial news on CNN, the importance of this has never been clearer.  Many of the conversations at PRSA08 have been revolving around the fact that proper PR Measurement is more important than ever.  Shonali herself argues this in a well argued guest post over at Kami Huyse's blog, Communications Overtones. It reinforces the points that Metrica has been making for some time in posts covering how PR can beat the credit crunch.

For the readers of Metrica's Measurement Matters who work in PR consultancy, I also thought it useful to draw your attention to a recent post by Dr Tom Watson, 'How PR agencies survive a recession'. Dr Watson is an acknowledged authority in the field of PR research and measurement, but also brings a serious commercial angle to his thoughts having run a PR agency for over 18 years.  He lists 14 'recession management action points.' 

One of these is particularly relevant:

I'm en-route to 'The Point of Connection' this weekend, the PRSA international public relations conference in Detroit.  Having arrived late yesterday and unable to get a direct flight, I am staying over in Chicago. 

I was surprised to find literally around the corner from my hotel, the ABC7 Chicago television station.  The news studio is at street level and has a massive glass frontage.  As well as watching the presenters doing their stuff, there's also a display highlighting the station's history, milestones, key presenters (including Oprah) and changing technology over time.

Watching the presenters at work, and the hi-tec cameras recording them, made me think about the people watching them and wondering how they were doing this.  Returning to my room, I could see the same presenters on my TV.  And on my laptop.  And on my mobile phone.  And it got me thinking about how the world of media is converging and moving away from a single specialised platform to multiple platforms.  Journalists are becoming broadcasters.  Broadcasters are becoming web content providers, and websites are becoming more like TV stations.

This all ties in to a point made on Radio 4's 'The Media Show' which I listened to via its podcast focusing on Channel 4's decision to abandon DAB (digital radio) as a platform.  The problem for Channel 4 was not that the quality, or the reach of the broadcast signal, but that dedicated DAB radios have not sold enough sets in the UK and therefore are unable to deliver large enough audiences to make it commercially viable.  According to one of the guests, there is a very real danger that DAB radios in the future will only provide BBC content; all commercial broadcasters will have abandoned it. They didn't argue that digital radio technology was dead, just that single platform delivery systems (like a DAB radio) are not what today's consumer wants.

As the speed of technology continues to revolutionise the media, we need to bear in mind that it's not just the new technologies that matter, but the ability of the platforms to carry them that will dictate their importance and ultimately survival.

One final thought, as I have typed this, the Oprah show has come on the TV.  Oprah's guest today is Jeff Bezos, Amazon's founder.  And no wonder as Oprah has just announced that Amazon's Kindle is her favourite new gadget.  I love the idea of the Kindle, but I love it as a technology.  As a platform, it only allows me to read papers and books.  What about listening to music and surfing the web?  When it becomes a multi-delivery platform I will be in the market for one.  Until then, I suspect it will be just the early adopters, which incidentally, if you would like to join, Oprah has a $50 discount code valid for 7 days.

Most of you are fully aware of social media but to many of our clients it is the new kid on the block, so to speak.

Metrica works with many clients that have celebrity brand association, for example James Brown London. This is a good example of a client that is seeing bloggers fast becoming as influential in their field as most journalists. Mario Lavandeira, AKA Perez Hilton is renowned as one of the most powerful and most recognised names amongst celebrity gossip. With an average of 198 million page views a month (according to Quantcast) and advertising space on his homepage fetching up to a staggering $54,000 a day. 

Celebrities fear the wrath of this blogger who started his site from a laptop in a coffee shop. He has his favourite celebs and then there are those who dread his comments. Perez Hilton was voted second on the ‘Web Celeb 25’ in 2007. He is fast becoming a celebrity in his own right, no longer simply invited to press events, he now walks the carpet with the other celebs and is frequently cited as a source in UK gossip magazines.  

So how do PRs measure the impact of blogs such as Perez’, on their clients?  At Metrica, and particularly in the consumer team to which I belong, we firmly believe in normalising social media and using appropriate measures in much the same way as the rest of your media coverage. Daunted by the ever increasing number of blogs? Similarly to having key media lists in traditional media, a good way to start measuring blogs and the impact they have on your audience is by tracking a select list of influential blogs to your client.  As the consumption of conventional media is declining, it is even more important to focus on the quality and impact of social media. 

In the same way that a PR would reach out to a journalist who was making negative remarks about their client/brand, they should do the same with a blogger. Perez Hilton was continuously making negative remarks about Victoria Beckham, so she went and had lunch with him!  She showed him that his perception of her was completely not true and since then she has been one of his ‘favourite’ celebrities.  

Whilst social media is new to some people, it doesn’t have to be frightening.

In many industries for many years, automation has replaced old fashioned human labour and brought efficiencies and cost savings.  But while computers can be faster and more accurate than humans at many things, will they ever do what human beings do best?  Will computers ever think?  In 1950, mathematician Alan Turing wrote that “if, during a text-based conversation, a machine is indistinguishable from a human, then it could be said to be thinking”  He devised his famous Turing Test and predicted that by 2000 computer technology would have advanced so much that 30% of human judges would be fooled into thinking that a machine was human.

So far the Turing Test has never been passed.  Over last weekend, controversial scientist Kevin Warwick conducted the latest set of tests as part of the 18^th Loebner Prize for artificial intelligence.  Unfortunately even the winning entry, a robot called Elbot, managed to convince 25% of the judges and so officially failed the test.  “We really, really have come very close” said Warwick, although it should be noted that the sample size in a field made up of computer experts and journalists was just 12!

When it comes to media analysis, while computers have helped enormously,  full automation has proven to be difficult given the complex task of “understanding” language and therefore attributing the correct sentiment, messages and topics.  This has resulted in the emphasis on a human analyst to read every article which means that the cost of an analysis program scales proportionately to the volume of coverage.  When you are interested in measuring your own organisation in the relatively small universe of traditional media this is fine but what if you want to benchmark yourselves against competitors.  What if you want to extend it to social-media - how do you measure the whole of the internet?  Since the cost / benefit equation becomes an issue, maybe it is time to look at automation again.

Indeed, many of the social-media specialist companies have gone down the automation route.  In Nathan Gilliatt’s guide to Social Media Analysis 2008, at least 23 out of the 64 companies featured use some form of automated analysis.

So just how good have computers got?  It is certainly true to say that there have been significant developments over the past few years, particularly in the areas of natural language processing and machine learning.  Topic identification and entity extraction has been widely researched, while measuring sentiment has also received a fair amount of attention.  In a series of experiments between 2002 and 2004, Bo Pang and Lillian Lee from Cornell University used a variety of machine learning techniques on positive and negative film reviews.  They managed to produce accuracy rates of between 75% and 86%.

This of course can be applied to measuring sentiment for organisations in media coverage.  A number of generic tools exist for doing exactly that with similar results.  For example Corpora Software (who were bought by Infonic and are now part of Lexalytics) have claimed a accuracy of 75%.

One problem with sentiment is that what is favourable in one domain is not necessarily favourable in another.  For example while the word ‘cancer’ would usually be an indicator of negative sentiment, for a cancer charity this would probably not be the case.  Domain specific machine learning can help with this issue.  We recently conducted an experiment on coverage from a leading computer anti-virus company.  When using a generic sentiment tool we managed to get an accuracy of 79%.  However, many words that would usually be considered negative: ‘virus’, ‘attack’, ‘malware’ are often found in positive articles.  By ‘training’ the sentiment model on more relevant coverage we managed to increase the accuracy to 92%!

So we know how good computers are, but how good are humans?  Copora reckoned that there was a 82% chance of two or more human analysts agreeing with each other while in a piece of research from the Natural Language Processing team at Microsoft, human analysts agreed on average just 74% of the time.  To quote Microsoft: “this suggests that the task of sentiment classification is difficult even for people”.  From our own experience, this is pretty low – with a tightly defined brief, a good system that reduces subjectivity backed up with extensive quality control will result in much better accuracy and certainly better than we can get with a computer.

US measurement expert Katie Paine has said “We’ve gotten very good at teaching computers to understand words, the problem is that they don’t understand the nuances of conversations. Computers still can’t tell the difference between sarcasm and irony. And throw in slang and you have an even bigger problem. Facebook, HP and Microsoft did extensive research before selecting measurement tools and all three insisted on human analysts. So I ask you if some of the leading players in technology don’t trust computers, why should you?”

What about my own thoughts?  Are computers as good as humans for analysis – no.  Would I trust a computer to measure my most important media – no.  However for getting a litmus test on the ‘long tail’ be it social media or competitor benchmarking then there is a definite use for automated analysis.  Finally I feel that this is not a black and white issue – it is not a straight choice of computers vs humans.  Why not get the best of both worlds with a hybrid model.  An emphasis on humans for the important stuff with more automation combined with a certain amount of human checking for the rest.

As to the future, as technology advances we may well see the removal of humans from the analysis ‘engine’.  However since even the mighty Alan Turing got his prediction wrong, you might not want to bet on this.

Sometimes you just have to let it all out... Here's an interesting rant about ROI by Gary Vaynerchuck. He talks directly to the marketing and advertising community, but I think what he has to say is relevant to PR.

In a stalling economy, where marketing and PR budgets may suffer, you need to get more bang for your buck. You need to maximize your Return On Investment.

Major newspapers and magazines are suffering declining readerships, though advertising in them can still cost a small fortune. Audiences are becoming more and more fragmented as the number of media outlets and formats grow: it's getting harder to reach them through traditional channels.

However, every cloud has a silver lining. The eyeballs haven't gone away, they've simply moved to other media channels. There's a major movement to online media and also of course to social media. The fragmentation of the audience actually provides those in advertising, marketing and PR with the ability to more precisely identify, target and reach the right audience.

Going online, undertaking sophisticated pr planning, carefully researching and targeting the correct audiences and focusing your spending and campaigns better will provide optimum 'bang for buck' and ultimately a great ROI.