Scholarship with Impact

Abstract

This essay offers a personal and therefore partial view of the possibilities of scholarship with impact. The challenge of scholarly impact is framed in the duality of scholarly and policy/practice impact. This is an aspiration that not all social scientists interested in the field of management may aspire to. However, at a time when the British Academy of Management (BAM) is celebrating its 25th anniversary, it may be appropriate for some members of our local and international communities to raise their aspirations and deliver forms and processes of knowledge which meet the double hurdle of scholarly quality and policy/practice impact (Pettigrew, 1997, 2001). The paper sets out to accomplish this task in the following way. The first section outlines some of the current instrumentalities facing UK scholars of management in and around the 'impact agenda'. Of particular interest here are the recently published Decisions on Assessing Research Impact, published by Higher Education Funding Council of England (HEFCE, 2011) for the Research Excellence Framework (REF) 2014, and the work on impact and assessment measurement carried out by the Economic and Social Research Council Evaluation Committee and published in 2009 and 2011 (ESRC, 2009, 2011). The following section then seeks to contextualize the present instrumentalities by locating the impact agenda in the much wider, deeper and historically embedded debates on the purposes and practices of social science and management research. Reference is made here to some of the influences on current UK management research and the way our fields of knowledge are themselves a product of wider pressures to change the character of knowledge and how it is produced. The third section deepens our treatment of impact by examining the progressive influence of the ESRC Commission on Management Research (ESRC, 1994); the British Journal of Management Special Issue (Hodgkinson, 2001); the work of the ESRC (2009, 2011); and pressure from scholars on both sides of the Atlantic, e.g. Hambrick (1994), Pettigrew (1997, 2001), Starkey and Madan (2001) and Van de Ven (2007). In the fourth section some of the determinants of research impact in the sphere of policy and practice are discussed. The fifth and final section then offers a conclusion. The recent announcement by HEFCE (2011) that 'there will be an explicit element to assess the impact arising from excellent research, alongside the outputs and environmental elements' is a notable step forward in national policy and a clear statement of intent for the future. The weighting of 20% for impact alongside the 65% for outputs and 15% for research environment is a compromise for those who fought hard to resist the idea of assessing impact. But the change has now been made and there is a clear signal that the 20% will be increased in subsequent exercises. The REF and its predecessor the Research Assessment Exercise (RAE) perform multiple functions. They are at once a resource allocation mechanism to reward high research performers and admonish the lesser performers; a quality assessment mechanism; and crucially a potential mechanism of culture change for all the academic communities in the UK higher education system. We as scholars are fond of our scholarly routines and those routines take much shifting even with pressure from HEFCE. In my view, successive RAE assessments right back to the mid-1980s have changed individual and institutional behaviour. Scholars are now much more attuned to where, when and how they publish. The spotlight of performance has become an uncomfortable zone for individual, group and institutional behaviour. But the stringency of the audit society (Power, 1999) has had positive and negative effects. The RAE has added to other career, professional and self-generated pressures and may have created a displacement of goals. For some publishing may have become an end in itself. Including the impact of research in the social, economic and cultural spheres beyond academia is an important corrective to this displacement of goals. In shifting the focus from the intermediate good of published outputs to the final good of scholarly impact (in the outputs criteria) and policy/practice impact in the impact criteria, there may indeed be important behavioural and cultural changes to our practice as scholars. Enhancing scholarly quality remains essential but in time this opening up of the impact agenda can create greater legitimacy for a portfolio approach to publishing for those scholars deadened by the retreat to defining scholarship just in terms of publication in A-rated scholarly journals. Of course, our position in business and management schools means we have many possible links out into stakeholders in the economic, industrial and public policy spheres of society. We should not be threatened by the impact agenda but colleagues in the arts and humanities most certainly have been. Stefan Collini writing in The Times Literary Supplement, 13 November 2009, commented at the end of his forthright article that unless the REF guidelines are changed 'Scholars in British Universities … will become "door to door" salesmen for vulgarised versions of their increasingly market orientated products'. In a similar vein, James Ladyman in the Oxford Magazine No. 294 observed that 'It is astonishing that the ostensibly intelligent and knowledgeable people who are responsible for higher education policy should evince the combination of philistinism and ignorance responsible for the ridiculous and deplorable ideology of impact now being foisted upon us' (2009, p. 4). As we all know, one person's evolutionary change is another's revolutionary change. The impact agenda, which, of course, is not just confined to the REF but also now shapes the applications to and evaluations of Research Council grants and awards, will remain controversial even for some living within professional schools. But all changes have their antecedents and it is to those germane to our management research communities that I now turn. Interest in the impact of social science and management knowledge is as old as the fields of study themselves. Current UK instrumentalities are but a moment in a long process whose pathway of development is unpredictable and shaped by many stakeholder interests. For most of the 20th century, social scientists have debated 'knowledge for what purpose' (Lynd, 1939) and, indeed, for whose benefit? Current discussions of impact are embedded in the thesis of a change in the character of knowledge and assumptions about a new process for producing knowledge. This is a not a place to review such development but the interested reader may wish to explore Jasanoff (1990, 2006), British Academy (2004) and Nowotny, Scott and Gibbons (2001). Elsewhere (Pettigrew, 2001), I have argued that the elements of the change are many, but the most often debated features include a more porous boundary between science and society; a resultant loss of research autonomy; a breakdown of assumptions about unitary views of science and linear notions of the scientific process; a greater range of participants in the knowledge development process and greater pluralism of research practice; a greater recognition of the localized (in time and space) character of research practice and outcomes; and a recognition of the complex interactions between multiple stakeholders in the research process and a more contested landscape for evaluating the quality and relevance of research processes, outputs and outcomes. These hypothesized and real changes have influenced and are influencing us as a management research community. Colleagues recruited into business and management schools in the last 18 years conceivably may have missed the report of the ESRC Commission on Management Research which was published in 1994 (ESRC, 1994). This Commission was chaired by George Bain, then Principal of London Business School, and included visible BAM members such as Arthur Francis, Andrew Thomson, Robin Wensley and myself. The title of the Commission's report Building Partnerships: Enhancing the Quality of Management Research well signals its core theme. Published as it was seven years after the creation of BAM, the report's findings provide a barometer of the perceived state of management research in the UK at that time. The report noted the rise and rise of business and management schools in the UK higher education system (a trend that has been sustained), but also acknowledged that the expansion had brought its problems as well as its successes. 'Rising concern about the character, quality, funding and relevance of management research led to a series of meetings between the ESRC and the management research community and, in particular, with the British Academy of Management' (ESRC, 1994, p. 1). Looking back, it is amazing how prophetic and measured the report was. Although in the contemporary sense the language of impact did not permeate the document, the lack of impact of management research was a key part of the diagnosis. 'One of our themes is that management research still does not make enough impact on users and thus on management practice. But its capacity to do so is not in doubt' (ESRC, 1994, p. 5). Notwithstanding stereotypes about the pervasiveness of implementation gaps in public policy, many of the recommendations of the report did in time happen. Thus we find that the ESRC, BAM and the Foundation for Management Education (FME) have supported investments in fellowships to develop scholars; the same bodies have greatly supported research networks in the management research community; and the ESRC has included relevance to users in the grant application and evaluation processes. Crucially, the £30m ESRC/Engineering and Physical Sciences Research Council management research initiative (Advanced Institute of Management (AIM) Research) has happened and has benefited the development of many UK management scholars. It is also noticeable that recommendation 9 of the report anticipates aspects of the new REF, although 'relevance' was preferred at that stage to 'impact' as the criterion of worth. 'We recommend to the Higher Education Funding Councils that they incorporate the assessment of research relevance alongside their periodic assessments of research quality' (ESRC, 1994, p. viii). The only important recommendation of the ESRC Commission which did not happen was the creation of a Management Research Forum whose task was to have been to facilitate agenda building, knowledge exchange and constructive partnerships between researcher and user communities. It seems the ESRC and other interested parties could find neither a very senior industrial figure to chair this body or the right calibre of executives and policy-makers to populate it. So our community problem of the management of indifference in the market place of potential users was real enough in 1994. I wonder if that situation has changed much in the last 20 years? It seems we remain challenged by the diffuse nature of the communities who could be our potential collaborators and users. There seems to be no natural focused community for our management research and the challenge remains a local one as we engage project by project (Adler, Shani and Styhre, 2004; Beech, MacIntosh and MacLean, 2010; Pettigrew, 2003). The ESRC Commission on Management Research was foresighted and delivered within its remit. Subsequently BAM and the FME have played their part in encouraging intellectual debate about the character of management research in the UK, with Tranfield and Starkey (1998), Starkey and Madan (2001) and the Special Issue of the British Journal of Management (Hodgkinson, 2001), which offered a UK and US academic commentary on the Starkey and Madan report. The report itself was a valuable contribution to an issue where there are no easy answers. The British Journal of Managment Special Issue revealed disparate views within the UK and between the UK and US scholars about the nature and purposes of business schools and universities and the varieties of knowledge and knowledge processes open to us, and helped to unpack easily used but complex terms such as relevance, user, collaborative research and Mode 1 and 2 research. But in 2001 the impact word had not yet captured the academic and political space. This language capture began to happen in the UK around the period 2005 onwards as the ESRC Evaluation Committee started to pick up the issue of assessing the non-academic impact of social science research. One of the material effects of this work has been the inclusion of non-academic impact assessment in the 2014 REF and, indeed, the form and process of assessment in that REF. Over the last several years the ESRC's Research Evaluation Committee (now Evaluation Committee) has been at the forefront of developing impact evaluation practice. Those scholars interested in clear statements of the progressive development of such work may wish to read two papers published on the ESRC Society Today website. The papers were respectively published in 2009 and 2011 and are titled 'Taking Stock' and 'Branching Out'. Margaret Macadam, one of the officers of the ESRC, played a key role in preparing these papers. Important related work includes Davies, Nutley and Walter (2005), Nutley, Walter and Davies (2007) and Meagher, Lyall and Nutley (2008). The first symposium organized by the ESRC in this stream of work was in May 2005. The report of this symposium by Davies, Nutley and Walter (2005) was an important influence shaping the principles behind subsequent developments. Key messages from this symposium included the recognition that different types of research produce different types of knowledge; non-academic impacts are diverse and can be stimulated by a number of different mechanisms; research contexts can be facilitators and inhibitors of research impact; and the assessment of research impact has to be multidimensional and draw upon both qualitative and quantitative techniques. There was also the clear message that at that time the RAE was a major barrier to non-academic research impact. In subsequent thinking the term non-academic impact has been substituted by policy/practice impact and the framing of policy/practice impact has always been addressed within the double hurdle of scholarly and policy impact. Earlier work on knowledge creep by Weiss (1980) reinforced later messages about the time lag and attribution problems in connecting research processes and outcomes with policy and practice impacts. A further progressive step was the recognition that the fate of knowledge was not just tied to receptive and non-receptive contexts (Pettigrew, Ferlie and McKee, 1992), but also to the tripartite relationship between the content of research, its context and the knowledge production process itself (Pettigrew, 1987). The explicit recognition that processes of knowledge production might matter made it much easier to incorporate recent thinking about the co-production of knowledge (Jasanoff, 2006). With this came the working hypothesis that early and continuing engagement between knowledge producers inside and outside universities increases the probability of policy/practice impact. The ESRC impact evaluation work has not just stopped with developing impact evaluation methods, including the possibility of identifying the economic benefits of social science research. The Evaluation Committee work has also explored the key drivers or determinants of impact and thereby how impact may be achieved. For this work it was even more important to be able to define conceptually and operationally what research impact on policy and practice is. Here unpublished work by Laura Meagher of the Technology Development Group turned out to be valuable (2009). Meagher identified five categories of impact: instrumental conceptual capacity building cultural change, and enduring connectivity impacts Instrumental impacts were defined as tangible products or services taken up by companies, policy-makers and practitioners. Conceptual impacts included generating new understanding or raising awareness among potential users of research findings. In my experience, conceptual impacts can also include reframing policy issues in order that new policy options and pathways to implementation open up or close down in a policy domain. This closing down of policy options through intellectual challenge can be a very important scholarly contribution, especially when robustly evidence based. Capacity building impacts include training and/or developing collaborative activities and cultural changes, and enduring connectivity impacts can shape both willingness to engage in knowledge exchange activities and the establishment of sustainable relationships between knowledge producers in and outside universities. As we all know, the manifest payoff from this ESRC Evaluation Committee work will be real understanding of why and how policy/practice impacts occur when they do. Here there are early signs of illumination which will help us to shape a practical theory to inform the development of the capabilities, practices and processes to increase the range of scholars doing high impact research. Research in any field is a probabilistic activity. There are few predetermined processes and outcomes in creative activities such as scholarly research. Aspirations that researchers may have to influence policy and practice outcomes are themselves complex influence and change processes and we know from the established research literature on change that such changes are much shaped by the context, content and process of the changes themselves (Pettigrew, 1985, 1987, 2002). Encouraged by the report from their first symposium on non-academic impact (Davies, Nutley and Walter, 2005), the ESRC funded a series of case studies to examine why and how policy and practice impact occurs. Subsequent analysis (ESRC, 2009) has explored the determinants of impact by using the framework which explores the interdependent effects of research context, content and process. Thus the ESRC's impact evaluation work (ESRC, 2009) identified a set of key factors that appeared vital for impact generation. These include established relationships and networks with user communities involving users at all stages of research well-planned user engagement and knowledge exchange strategies portfolios of sustained research activities that build reputations with research users good research infrastructure and management support for user and knowledge exchanges, and where appropriate, the involvement of intermediaries and knowledge brokers as translators, amplifiers and network providers. These determinants of policy and practice impact are both contextually and temporally embedded. The history of knowledge production in the natural, medical and social sciences is full of examples where societal, religious and policy contexts have presented non-receptive settings for new ideas and those ideas have had to wait for the emergence of more receptive contexts for more developed recognition. But social scientists do not need just to wait until more receptive contexts emerge. If they have pre-existing relationships and credibility in policy arenas it may be possible to shape the climate for new knowledge to find acceptance. Serendipity can also act as a determinant of impact. Research findings may have the good fortune to coincide with the rise of policy initiatives. So one enabling feature of impact is the extent to which the content, originality and timing of ideas and new knowledge fits the rise and fall of issues in the policy context. On the process side, the ESRC impact case studies demonstrate that 'the most important factor contributing to the generation of policy and practice impact was the pre-existence of networks and relationships with research users' (ESRC, 2009). Such pre-existing relationships are most likely to be found in research centres and networks which pursue big themes well connected to past, present and future policy agendas. So sustainable research communities which pursue big themes over time may have big advantages in nourishing some of the conditions for impact, but only if the content of their work is matched or can be made to match with policy contexts and the community of scholars is wilful and skilful enough to want to have a policy and practice impact. The message here for experienced and less experienced scholars is that alongside the development of intellectual capital there is a big need to cultivate complementary, social and reputational capital. Like it or not, accept it or not, the combination of personal branding of the scholar and institutional branding of the academic base of the scholar all add to the potential of impact but cannot in themselves guarantee it. Although it is clear that dissemination is not in itself impact, those scholars who adopt a portfolio model of publishing and thereby reach out to many stakeholders have a further advantage. Skill in communicating ideas with conviction face-to-face is also bound to be important. We are all capable of accomplishments in the influence business in and around our research activities. These messages about some of the determinants of policy and practice impact raise important issues about the development of skills, knowledge and motivation for impact in our research communities. The good sign here is that cooperative strategies between BAM, the ESRC and AIM are bringing forward a research development initiative in this area called 'Building Capacity for Business Engagement with Impact'. Led by Richard Thorpe of Leeds and BAM and Mike Wallace of Cardiff and AIM, the idea is to run a series of workshops and training events to foster skills in the co-production of knowledge with high practical impact. Regrettably as yet, neither the ESRC nor many of our business and management school communities have felt it necessary to re-define doctoral education to allow access to the necessary skills and knowledge at the point when new scholars are being socialized into their respective research fields. But perhaps in time this will come. The old concern for knowledge for what purpose and its newest embellishment in aspirations for the policy and practice impact of social science and management research are general issues of potentially universal appeal and interest. In the context of this Special Issue of the British Journal of Management commemorating the 25th anniversary of the foundation of BAM, I have concentrated my observations on the UK context for management research. May I now very briefly redress that by pointing to, for example, the recent report on impact of research by the US body, the Association to Advance Collegiate Schools of Business (AACSB, 2008), and the May 2009 Special Issue of the Journal of Management Studies on theory and practice in management research. The AACSB report presented the findings of a task force almost exclusively made up of US-based scholars and drawing almost exclusively on US-based management research. As an accrediting body based in the USA but with wider international reach, the AACSB was particularly interested in the missions and the character of research in business schools and their importance for the reception given to current research outcomes. With this broader brief the AACSB looked more widely than the ESRC and in particular gave attention to the impact of current management research on management education and the general issue of the transfer of research into the classroom. The report's recommendations, like the much earlier ESRC Commission on Management Research (1994), sought to give emphasis to the linkages between academic research and practice; the links between scholarly enquiry and education; and the availability of more diverse business school mission statements and incentive systems which would encourage wider engagement between business school academics and their potential audiences and users. The importance of the Special Issue of the Journal of Management Studies (vol. 46, 2009) on management research is that it updates many of the debates rehearsed in the equivalently themed British Journal of Management Special Issue of 2001. The interested reader may find some encouragement in the writings of Hodgkinson and Rousseau (2009) and Starkey, Hatchuel and Tempest (2009) and the scepticism of Kieser and Leiner (2009, 2010). The debates will go on and on. Big changes in scholarly routines require many complementary points of intervention and consistency and persistency of support for change. The ESRC Commission on Management Research, the work of the ESRC Evaluation Committee, BAM, the UK FME and the Society of Advancement of Management Studies have in their different ways all helped to question, challenge and enable. The new REF 2014 ought to have a major impact by changing the criteria of worth in UK social science and management research and thereby incentivising change. But in the end real change will only occur by scholars changing their scholarly routines. Here I remain confident that further new possibilities to widen and deepen our concept and practices of scholarship after modernism will also help. There is now greater recognition that scholars can both continue to search for general truths and give greater weight to temporal and spatial context. The abstractionism and reductionism of the variables paradigm can be accomplished alongside a new concern for holism and sensitivity to action, dynamics, context and complexity. It is possible to temper the deductive ambition and encourage inductive thinking and research practice. Furthermore, this more contextual and dynamic view of knowing will open doors to all the other potential stakeholders of management research. All these dualities and extending boundaries of knowledge production processes will create some of the conditions for greater scholarly and policy/practice impact. Andrew Pettigrew is Professor of Strategy and Organisation at the Saïd Business School, University of Oxford, and Senior Golding Fellow, Brasenose College. Andrew is a co-founder and former President and Chairman of the British Academy of Management. His research has pioneered the use of contextual and temporal analyses of organizational processes of strategy making, change, decision making and power. He is the only non-North American to be made a distinguished scholar of the Academy of Management. In 2003 he was made a Fellow of the British Academy. He is also a Fellow of the Academy of Management and the British Academy of Management. He was awarded an OBE in the 2009 New Year's Honours list for services to higher education.