Working on large international PR measurement programmes for firms head-quartered in the US provides me with many benefits, not least the opportunity to network with some of America's smartest operators in the field of public relations measurement, social media and market research. One of the these is Jason Falls who writes an excellent blog 'Social Media Explorer.'

Today Jason has posted on an issue close to my heart - whether to rely on automated sentiment analysis when undertaking social media evaluation. The issue has arisen due to the overwhelming volume of content that the social web generates. Companies know that it contains much valuable information, but PR analysis providers are unable to scale up to measure the content with human coders in the same way that they always have done with traditional media.  Either they don't have access to enough people, or if they do, the costs incurred for the clients would be simply prohibitive.

As a result many firms attempting to measure content in social media have developed automated systems to assess the tone of the myriad of conversations and in some cases to rank posts, blogs, tweets etc for their levels of influence and engagement. A lot of these companies are software specialists who are branching out into new fields.  Social media content has brought their world colliding with that of PR and evaluation. Many have built flashy dashboards that appear to offer great functionality and amazing insights from all of this content.  

The problem is that the data that they are serving up is often plain wrong. Jason's post describes the issue in great detail so I would urge you to take a look at it.

At Metrica we have been helping clients to understand the threats and the opportunities that social media offers since 2006 (and analysing the traditional media since the early days in 1993!) Our approach to social media measurement has always been to use an element of automation, but to undertake large scale human checking on the data before providing it to our clients. The automation provides an element of filtering if you like, the humans provide the real business insights and the intelligence.  Metrica wouldn't base any business decisions on data that stands a significant risk of being flawed.  So why should our clients?  

Regular readers will know that along with Durrants and Gorkana, we are in the process of integrating our businesses to provide the PR industry with the very best media intelligence service available. As we do this, you can rest assured that human quality control, assessment and consultancy will remain at the heart of our offering. After all, as one client in the United States commented to me: "garbage in... garbage out."  Neither Jason nor I could have summarised the risks of relying on automated social media measurement better.