Imagine, if you will, this scene: 

Your CEO: "In this current climate, we need to make sure that all areas of the organisation are fully accountable for every £/$  spent.  Tell me, how is the PR function doing at the moment?"

You:  "We're working with a PR measurement supplier to help us understand that.  According to our latest PR metrics, we achieved 1328 PR measurement weighted impact rating points last month."

CEO: "You what?! Says who?"

You: "Our media evaluation firm tells me.  They say this is a really good score compared to their other clients."

CEO: "OK.  How did we do last month?"

You: "We got 1392 points."

CEO: "So we're 64 points down in a month.  Should we be concerned?  Why are we down?  Where did these points go, and what are you doing about it?  In fact, how exactly is this 'sophisticated PR measurement weighted impact rating index' devised?  Is it tailored for us, does it reflect our industry, and is it linked to our business objectives? "

You: "Errr...."

One of the recurring themes at this week's Measurement Summit was the need for the public relations industry to provide sophisticated measurement, especially now in the current economic climate.  PR evaluation needs to be tied to your organisation's objectives, whether these be to increase sales, raise awareness, shift perceptions (a combination of these or something else altogether).  Barry Legetter, AMEC's executive director, and a fellow delegate at the Measurement Summit has posted about this on his blog the Thoughtful Thud.

So I am particularly concerned at the number of new scoring systems that are being heavily promoted in the market place.  Most of these are being promoted by either press clipping agencies who are trying to grab market share in the measurement field or CRM based software companies with the same motives.  All of these scoring systems are positioned as being highly sophisticated and as the answer to all your measurement needs.  Most of them have fancy names, fancy descriptions and in some cases fancy acronyms. 

In fact, at the conference, the latest has been launched - Verismo from Cymphony.  Versismo, like most of the scoring systems is actually based on some good ideas.  it's the way that is is executed that worries me.  TNS / Cymphony marketing material states that it is based on:

Visibility - the reach against your target audience, prominence of mention, headline mentions, focal point, spokespeople and 'etc'

Reputation - by which they mean message delivery, brand positioning, competitive comparison and 'etc'

Influence - media source quality and credibility, journalist credibility, stature list priority and 'etc'

Sentiment - the favourability of the article, the automated sentiment, consumer perspective, editorial perspective and 'etc'.

All of these measures (including the 'etcs' I can only presume) are fed into a model which apparently "weights data points to model achievement of your key communication goals".  These weights can be adjusted to achieve the optimal results.  Once everyone is happy, the data is output as the VERISMO score.  Which we will all be relieved to know is a language that "all stakeholders can understand". 

In my 20 years experience in the PR industry where I have worked at PR agencies, in-house and, for the last 13 years, running a media analysis business, I am yet to see a scoring system that is appropriate to all clients, fool-proof, accurate, meaningful or easily understandable by all stakeholders.  (For interest, Metrica used to produce our own weighted impact score.  Although we believed it to be incredibly sophisticated we soon realised that it had some basic flaws and it ran the risk of making our results difficult to interpret, and sometimes plain wrong.  We stopped offering it in the mid 1990s, and have not looked back as a company since.) 

These days we are often to be found being called in by companies asking us to make sense of analysis that they have been given by a competitor using a scoring system.  I think of the time that a major chemist in the UK asked us to explain why a competitor's scoring system would not increase in value despite coverage in Cosmopolitan leading to one of their brands selling out nationwide.  The reason was that this particular system didn't attribute the article any 'impact score' because it was a) small, b) on page 92 and c) on a left hand page.  These factors combined to make it next to worthless to the scoring system's weighting algorithm.  To the client on the other hand it was of course priceless.

In our opinion, PR evaluation scoring systems and indices should all be provided with a major health warning. 

• They are all proprietary meaning that no two companies systems are compatible.  This means that if you want to change supplier, you will lose all of your historic data.  This raises a concern as they lock you into one supplier.  
• Most of them utilise elements of a 'magic bullet' approach to pr measurement.  This means that your coverage will be interpreted in a similar way regardless of whether you are Greenpeace or Shell, Bank of America or the R.S.P.C.A.  For these reasons, the flaws in the scores as in the above example with the chemist are all too prevalent. 
• They mask the real data behind your results.  Think about it.  Any measurement system that weights a number of criteria into one score, by default has just masked the separate data points.  This is  all the more alarming when you consider the amount of measurement companies who use automated (i.e. computers) to read and analyse your clips.  Any mistakes that are made in the analysis process are 'hidden' by the scoring system. 
• They can't help with a sophisticated PR planning process. Your score's gone down/up?  Well what does that actually mean? 
• And finally they are often used to make the supplier sound sophisticated giving them a USP, and ultimately, they hope, helping to justify high prices.

At the end of the day, scoring systems and AVE's are basically bed fellows.  They both tend to generate large numbers which don't really mean anything.  Nor are they speaking the language of the board room and the C-Suite.  To do this, you should be focusing on measuring your PR achievements against both your PR objectives and your organisation's objectives.  Anything else is going to run the risk of leaving you skating on thin ice, and is the last thing you need right now. 

Footnote: I'm sorry to appear to pick on Cymphony's Verismo, it's nothing personal.  As stated in the post, it is only because this is just one that I am aware of that was launched recently.  Another that I have recently been made aware of is 'Ebiquity' from Billetts which according to its marketing material is going to 'revolutionise the PR and reputation management industry.' It's a massive claim which you can judge for yourselves here.