PR Evaluation Survey 2 - 45% say there's a "lack of information"

This is the second of two posts on small-scale research into current PR measurement and evaluation practices. See # below for the methodology.

The first posting identified that while 80% claimed to formally or informally evaluate PR activity, 43% continued to use AVE as a prominent measurement metric.

This post reports on who is doing evaluation, the percentage of PR activity evaluated, some attitudes, knowledge of the Barcelona Principles and students’ views on the whether evaluation was being undertaken ethically.

Who did the evaluation of PR activities?

Account team/PR staff – 45.6%

In-house research and outside media analysis company – 26.5%

In-house research section – 16.3%

·         It looks as if self-certification of PR activity is common

What percentage of PR activity was evaluated?

100% - 35.7%

90% - 19.0%

80% - 11.9%

·         Good news – Two-thirds of the 80% who evaluate their PR coverage (53.3%) measure and monitor 80% to 100% of their PR activity

Questions about attitudes to PR measurement and evaluation

1)    PR budget is difficult to obtain: 37.7% agree or strongly agree; 28.2% disagree or strong disagree

2)    There is a lack of information on PR evaluation: 45.3% agree or strongly agree; 34.0% disagree or strong disagree

3)    There is a lack of time for PR measurement: 55.6% agree or strong agree; 24.1% disagree or strongly disagree

4)    PR is difficult to measure: 35.8% agree or strongly agree; 35.8% disagree or strongly disagree

5)    Practitioners fear evaluation: 25.0% agree or strongly agree; 42.3% disagree or strongly disagree

6)    Without measurement, PR’s future is threatened: 64.1% agree or strong agree; 20.8% disagree or strongly disagree

·         Mixed messages – 7% more say budget is difficult to obtain; there is a gap of 11% between those who agree that there is a lack of information on PR measurement (a large 45.3%) and those who don’t; a clear majority don’t have time to evaluate ('too busy doing PR'); there is a balance between those who find PR activity difficult to measure and those who don’t, which is an improvement; Many disagree that practitioners fear evaluation, but nearly-two thirds (64.1%) agree that PR’s future is threatened without the consistent use of measurement and evaluation.

·         The most concerning attitudinal outcome is that 45.3% of organisations say there is a lack of information on PR measurement and evaluation methods. This is a negative comment on the professionalism of many practitioners who can’t be bothered to look at abundant resources in terms of online materials (often free), books and training courses. Measurement and evaluation has been a major education and training topic since the mid-1990s and appears to have been ignored by them.

Students were asked about the percentage of PR budgets that were applied to PR measurement and evaluation. Most, not surprisingly because of their junior positions, ‘Didn’t know’ (53.8%) but the next largest valid percentage was for 1-3% of total budget (17.3%), which aligns with other research in the UK and Australia.

Barcelona Principles

Students were questioned whether the Barcelona Declaration of Measurement Principles (AMEC 2010) was referred to or mentioned at their main placement. Their answers were wholly negative with 55/55 ticking “NO”. Bearing in mind the support that AMEC, CIPR, PRCA, PRSA, IPR, Global Alliance, etc have given to the Barcelona Principles in the past three years, this is a very disappointing result but is similar to US research (Ragan and others) that found low awareness.

Was PR evaluation undertaken ethically

YES – 74.0%; NO – 26.0%

No comment!

# Methodology: PR students at Bournemouth University were surveyed recently about their experiences of evaluation practices during their 2012/13 sandwich year placement. 55 students (85%) took part, voluntarily, in the self-completion survey. As all but one (98.2%) had been on placement for nine months or more in a single organisation, they can be considered valid observers of practices taking place around them or in which they participated. The data were analysed using SPSS which provided descriptive statistics, mainly frequencies. The data used in these posts is based on ‘Valid Percent’ which omits missing answers unless they are a large part of the sample.

PR's future: Sing along with Tammy Wynette

Ten days ago, I posted on the PR Moment blog calling for a divorce between publicity/PR and strategic communication #. This blog reviews 10 responses which came from the UK, USA, Canada and Sweden and a separate poll of PR Moment readers.
Also, that well-known Northumbrian blogger Stephen Waddington penned a response on the PR Moment blog with the catchy title of “Elementary my dear Watson”. I was honoured!

What surprised me was the consensus for the admittedly difficult to achieve “PR divorce” proposal. Several respondents were in some form of complete agreement. I needed an extra glass of wine to cope with the shock.

Nick Grant referred to the PR=Publicity problem as the ’elephant in the room’, while Craig Fleisher and Fraser Likely commented that there is a trend to a separate strategic communication practice and body of knowledge already.

Even the 26% level of support for the split, shown in the PR Moment poll, was a plus. PR research over many years has found that more than 80% of practitioner effort goes on media relations and publicity. So 74% rejection is a positive trend, and was predicted by Canadian commentator Fraser Likely.

It will be a Long March for change, so what next?

- Unhook strategic communication from Marcoms at all levels. As suggested before, encourage publicity and Marcoms practitioners to form their own associations or link to CIM (marketing professional bodies). As Jesper Falkheimer noted, this is already happening in Scandinavia.

- Bring together online resources about best practice and theory in strategic communication and make them accessible to managers. There are already academic journals and leading texts in the field, as well as conferences.

- In the interim, demand that PR bodies give more recognition to strategic communication in their awards; the standard of many awards is woefully poor because if their focus on junk data and public (a point made by Fraser Likely)

- Starting with the Global Alliance, get it to drop ’public relations’ from its title and focus more on the 'communication management'. This would encourage PR bodies to reorganise and retitle towards strategic communication. For instance, the Institute of Public Relations (IPR) could become the Institute for Strategic Communication, as that is the overwhelming theme of its work.

So let's sing along with Tammy Wynette for “D-I-V-O-R-C-E” for strategic communication from Marcoms/Publicity. (But not in our private lives).   # http://blog.prmoment.com/brand-publicity-and-strategic-communications-are-two-different-professions-says-bournemouth-universitys-prof-tom-watson/  

Valid and Reliable? Why PR experts may be wrong

This is a difficult blog to write. It has the potential to upset some really committed folks who put a lot of time into thinking about public relations. However, it has the possibility for a very positive outcome.

Some practitioners are kind enough to give guest lectures to students. Others (some the same people) write blogs. A few commit themselves to writing 250-300 pages of a book, and the publishing grind that goes with it. They are good people. And it’s almost a ‘golden age’ of expert practitioners contributing to PR and corpcomms development in the UK.

But they are missing out on two vital concepts that could make their commentaries and books even more valuable. They are ‘valid’ and ‘reliable’ which are key components for producing best practice and theory that can really advise practitioners and make PR more effective.

Here are some examples of the type of commentary and “theorising” that comes unstuck. The first was a challenge to Jim’s Grunig’s concept of “symmetrical communications” which is based wholly on anecdotes and personal observation. The Grunig concept is not without challenge in the academic world but case studies and structured research are applied, not anecdotes and assertions.

The second was a set of “brainstormed” set of six descriptions of communications operations prepared by a PR organisation without any evidence there had been any preparatory study (this is not a new topic) or consistent research method applied. Neither of these assertions has “validity” or “reliability” for application in either practice or theory. They may be insightful but are just informed guesses.

Why is theory valuable for PR? Essentially, it aids prediction: “A sound theory is the conceptual foundation for reliable knowledge; theories help us to explain and predict phenomena of interest to us and, therefore, to make intelligent practical decisions” (Frankfort-Nachmias & Nachmias 1992). The outcomes of applied theory are organization, explanation, prediction and greater control of the situation. All these factors help practitioners build campaigns and programmes which are more likely to achieve their objectives because they are built on a valid, reliable base.

Can practitioners develop commentary and assertions into theory? Yes they can, and it’s very productive. In Australia, evaluation expert and commentator Jim Macnamara developed high-standard academic research skills and is now a widely quoted expert (and latterly a professor). Another approach is the research partnership of Chicago consultancy boss Lou Williams and San Diego professor David Dozier who for many years have researched best practice in internal communications. Their work is highly rated. In the US, practice research is fostered by the Institute for Public Relations (see its excellent www.instituteforpr.org) which gets industry backing for the type of robust research that practitioners look up to.

In the UK, is it time for industry to support continuing research into practice by creating an equivalent to the Institute for Public Relations? Partnerships, with some funding, between academic and the PR experts will produce enduring results. I hope so because there is so much energy and enthusiasm to be applied that could result in ‘valid and reliable’ outcomes for the PR sector.