Monday, 4 March 2013

Valid and Reliable? Why PR experts may be wrong

This is a difficult blog to write. It has the potential to upset some really committed folks who put a lot of time into thinking about public relations. However, it has the possibility for a very positive outcome.

Some practitioners are kind enough to give guest lectures to students. Others (some the same people) write blogs. A few commit themselves to writing 250-300 pages of a book, and the publishing grind that goes with it. They are good people. And it’s almost a ‘golden age’ of expert practitioners contributing to PR and corpcomms development in the UK.

But they are missing out on two vital concepts that could make their commentaries and books even more valuable. They are ‘valid’ and ‘reliable’ which are key components for producing best practice and theory that can really advise practitioners and make PR more effective.

Here are some examples of the type of commentary and “theorising” that comes unstuck. The first was a challenge to Jim’s Grunig’s concept of “symmetrical communications” which is based wholly on anecdotes and personal observation. The Grunig concept is not without challenge in the academic world but case studies and structured research are applied, not anecdotes and assertions.

The second was a set of “brainstormed” set of six descriptions of communications operations prepared by a PR organisation without any evidence there had been any preparatory study (this is not a new topic) or consistent research method applied. Neither of these assertions has “validity” or “reliability” for application in either practice or theory. They may be insightful but are just informed guesses.

Why is theory valuable for PR? Essentially, it aids prediction: “A sound theory is the conceptual foundation for reliable knowledge; theories help us to explain and predict phenomena of interest to us and, therefore, to make intelligent practical decisions” (Frankfort-Nachmias & Nachmias 1992). The outcomes of applied theory are organization, explanation, prediction and greater control of the situation. All these factors help practitioners build campaigns and programmes which are more likely to achieve their objectives because they are built on a valid, reliable base.

Can practitioners develop commentary and assertions into theory? Yes they can, and it’s very productive. In Australia, evaluation expert and commentator Jim Macnamara developed high-standard academic research skills and is now a widely quoted expert (and latterly a professor). Another approach is the research partnership of Chicago consultancy boss Lou Williams and San Diego professor David Dozier who for many years have researched best practice in internal communications. Their work is highly rated. In the US, practice research is fostered by the Institute for Public Relations (see its excellent which gets industry backing for the type of robust research that practitioners look up to.

In the UK, is it time for industry to support continuing research into practice by creating an equivalent to the Institute for Public Relations? Partnerships, with some funding, between academic and the PR experts will produce enduring results. I hope so because there is so much energy and enthusiasm to be applied that could result in ‘valid and reliable’ outcomes for the PR sector.


  1. Interesting point Tom. I might say valid, but that would need empirical evidence it seems ;-)

    There are, as you know, two ways to move any subject forward. Empiricism and rationalism. The first, as you point out, is more expensive that the latter, and few parties appear to justify the investment. I believe the CIPR is intent on undertaking more research subject to budgets, so I'll flag your post up if you haven't done so already.

    I do not currently have the budgets to undertake empirical testing of my hypothesis, so I lean for now on logic. I'm not sure if your reference to a challenge to Grunig is a reference to my recent guest lecture at Bournemouth – if so, I can assure you that logic is more the basis than "anecdotes and personal observation". That and an understanding of complexity, a critical dimension distinctly absent in any discourse I've seen about the 4th model.

    1. Philip: Thanks for the comments. My comments weren’t personalised and, as I indicated, the contribution of knowledgeable practitioners like you is offering a ‘golden age’ of debate in public relations.

      Industry questioning of the four Grunig models and "symmetrical communication" is not new. Academics aren't all in awe of them either. But (and it is an important caveat), their supporters always point to Excellence Theory being based originally on an industry-funded study (IABC) of PR management in around 300 organisations. Thus it has high reliability and much validity. That reliability has been tested in subsequent iterations of Excellence Theory in reviews published in 1997, 2002 and the 2007 festschrift. Another festschrift on the Grunigs’ contribution to PR theory will be published mid-year by Routledge.

      Like you, I have long held doubts about the concept of "true" symmetry. Social media has further reduced its validity because its bilateral concept is now splintered. There are other theories to consider but it's time for new approaches to be considered.

      Your contribution to new theory is to put forward the Influence Flow hypotheses based on observation. These could be developed into applied theory through testing in a structured manner to get the type of validity and reliability that defines them as valuable for PR practice in the long term.

      As for industry funding, it's not just CIPR (and/or PRCA) that I am referring to. My view is that the PR industry as a whole does not invest in itself, despite its long-espoused desire for professional status.

      The IPR in the US, my comparison example, has had its financial ups-and-downs but has industry sponsors who have fostered research. The outcomes are published for the benefit of all practitioners. In my case, the biggest industry funding I have received came from the IPR via a Coca-Cola grant. And only last week, my research team got a grant for CSR research from the Arthur Page Center at Penn State. Its funding comes from industry, too. Nothing like that exists in the UK outside the research councils where the success rate for applications is 15%.

  2. I welcome your Blogpost and sentiment.
    It comes at a price for us all.
    In an area where we have a particular interest, the evaluation of Public Relations, we see much evidence of smoke and mirrors too.
    A universal framework is helpful such as the one we proposed in the Lisbon Theory.
    Its drawback (or is it an advantage) is that it quickly becomes a universal framework for measuring management activity as well.

    It posits: "From the perspective (v) of an entity (n) to what extent (e) is this object (o) significant (s).

    Thus one overcomes the old chestnuts of 'what is PR' and AVE's and ROI measures but it does insist that PR must have a perspective and for most academics and practitioners the perspective is that of the client.

    On my way back from today's lecture, I was pondering how it would resolve the Paul Noble and Tom Watson problem of ROI in your book on evaluation.

    Of course the return can be taken as a perspective. Thus one might ask 'derived from this item/corpus: from the perspective of the Board (v) of the organisation (n) to what extent (e) is the return on constituents' cultural appreciation (o) significant (s).

    If (e) is a Likert scale and (o) are agreed cultural values identified in programme objectives then significance is an absolute.

    Of course, one has to be able to refer to empirically tested theories such as the one offer by Bruno Amaral and me at Bled showing how relationships and constituent groups for round a nexus of values (subsequent - university budgets for PR research being what they are - tests have tended to use Google's Big Data Tools which is a much less expensive way of doing this kind of research).

    This year, I will have a computer programme that will offer this facility if researchers have an interest in using it.

  3. I've been thinking about this post since reading it initially on Monday and have a number of thoughts. The first is that one of the issues in PR is the lack of willingness of most practitioners to engage with an existing body of knowledge. The CIPR qualifications do have an academic basis, which introduces practitioners (many of whom are very experienced) to different perspectives, and encourage a critical perspective. Unfortunately, only a minority of members of CIPR study for such qualifications - and although those who seek Chartered Status may look at a body of knowledge, they do not seem to be challenged in the same way that happens if you study for a CIPR qualification or a Masters/PhD, for example.

    My second thought relates to this and supports Tom's observations that the wider industry does not invest in a robust body of knowledge. So not only is there little financial support for researching ideas underpinning strategic practice, there is almost zero recognition of the value of developing the cognitive competencies of those working in the occupation. Training courses (whether internal or external) and conceiving 'models' without the robustness of critical examination and testing, do not help enhance the credibility of PR, its claims to be a profession or the validity of its counsel.

    As a third point, the industry seems to be more interested in using research for promotional purposes than to improve analytical understanding and skills. The ideas proposed by consultancies (eg evaluation or planning models) are often used as a means of promoting their services, rather than founded on existing theory or specific research. Likewise, there is a lack of validity or reliability in PR award programmes. Again, the CIPR and PRCA surveys of the 'profession' really do not stand up to any depth of scrutiny - and indeed, lack any depth of analysis in the way they are reported. The impression once more is of managing PR's reputation. Similarly, 'PR surveys' are a synonym for publicity vehicles. Many practitioner books likewise appear to be about promoting the author(s) than expanding and reflecting on what has gone before and adding perspectives that are, or can be, further validated by critical examination.

    But, my final point is that there could be validity in the opinions and 'anecdotes' that are how experienced practitioners have always reflected on their expertise. Bernays and his contemporaries whose work we now analyse for their historical and contemporary relevance often did little more than share their views and cases. What is missing with modern versions of this approach is an understanding of qualitative research methodologies. When we talk about anecdote and case studies (in literature or practice), these do not reflect a robust analytical approach. That could draw on knowledge of narrative methods of research, for example.

    There is considerable potential for the PR world to benefit on both the academic and practitioner sides, by engaging with concepts of robust research methodology. This can however, relate to qualitative, and not simply quantitative statistical, analysis. And, we should hold up both academic and practitioner work to critical reflection so that together we can seek to improve both what we do, and our reputation for doing it.