Mixed Methods: Combining Expert Interviews, Cross-Impact Analysis and Scenario Development

The article depicts a mixed methodology case which uses a qualitative-quantitative-qualitative approach. The research described used qualitative work with expert interviews for data collection, a quantitative analysis of the interviews and then a qualitative method of final scenario development for analysing and presenting the results. The case is offered to demonstrate that the introduction of the quantitative step of a cross-impact-analysis, which gives a mixed methodology, was beneficial for the overall research leading to surprising results that could not have been achieved with only a qualitative approach. Having a quantitative analysis step in-between, which demonstrated the most frequent and consistent results out of a wide range of overall possibilities, helped reduce researcher bias, thereby increasing the credibility of the findings. The paper concludes that judiciously used mixed methodology in general, and this approach in particular, will give researchers using qualitative data collection a much stronger foundation in terms of the analysis and display of data.


Introduction
Mixed methods in social science research are defined as a technique that "mixes or combines quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study" (Johnson and Onwuegbuzie, 2004:17). In the last ten years there has been considerable interest raised in using mixed methodological designs for research (Creswell, 2003;Johnson and Onwuegbuzie, 2004;Creswell and Tashakkori, 2007;Gorard and Cook, 2007;Johnson et al., 2007;May, 2007;Tashakkori and Creswell, 2008;Symonds and Gorard, 2010). The argument is made that methodological pluralism or eclecticism enables researchers to increase both the scope and the level of possible analysis (Johnson and Onwuegbuzie, 2004). However, despite increasing acceptance of the concept of mixed methodology research, much research is still presented as either qualitative or quantitative (Hesse-Biber and Leavy, 2011;Patton, 2010;Smith, 2008;Thyer, 2010). The broader skill set that is required to apply qualitative and quantitative methods is impacted by the lack of training, which Plano Clark has been describing as the biggest barrier towards the application of mixed methods research (Plano Clark, 2005). Creswell and Plano Clark (2007) as well as Teddlie and Tashakkori (2009) have published handbooks looking to offer mixed designs, but the literature on applications and successful utilisation is still rare (Varum and Melo, 2010). It is this gap that this paper seeks to address. This paper will outline and discuss a specific mixed methodological example. The researcher chose to use scenario creation as the methodology of choice (Bea and Haas, 2004;Mietzner and Reger, 2005;Varum and Melo, 2010), but rather than a more common approach where qualitative data analysis is used as a foundation for qualitative scenario building (Brauers and Weber, 2006;Fink, 2001;List, 2003;Tulbure, 2004), this paper outlines the use of a quantitative data analysis stage. The strength of the technique is seen as reducing bias while adding credibility; the paper will assess both the strengths and the areas for development of this mixed method application.
Initially, the recent developments in mixed methodological thinking will be explained, before an outline of the methodological approach of the paper itself it proffered. The case example will then be described and analysed, before the implications of the example are given. The paper concludes that the use of this approach enables a more rigorous approach to scenario construction and strengthens this form of data collection and analysis.

Introduction to case
The case presented here is a theory excerpt using a former work towards a topic on demographic change and its implications on consumer, travel and leisure behaviour and education in the year 2020 (Muskat, 2008). Demographic change in this context describes the fact that many industrialised countries have a so-called sub-fertility rate of below 2.1 children per woman. Combined with an inefficient migration policy, these countries face an ageing population. This ageing average of a population has direct effects, such as an increase in the demand for health care for older people, or a decrease in demand to build new kindergartens. There are also indirect effects such as a gradual shift towards addressing older customers in advertising, using older models and advertising different products.
www.ejbrm.com 11 ISSN 1477-7029 These effects have different implications for different cohorts, e.g. for the Baby Boomers being the generation born after World War II until 1961, Generation X being born from 1961to 1976and Generation Y being born from 1976until 1991(cp. Coupland, 1991MacKay, 1997;Salt, 2006). One of the examined implications of the demographic change is the upcoming retirement of the Baby Boomer generation. They will retain their high spending power and will then have the time to spend it. Different travel and leisure behaviour was analysed to show different market segments within the travel market. Also an age effect vs. a cohort effect was discussed. This describes the fact that former older generations have travelled according to what seemed appropriate to their current age, while today"s older generation travel maintaining their acquired travel behaviour.
For the younger Generation X and Generation Y other aspects in life are more important e.g. tertiary education, family planning and the question of both male and female participation in the work force. A university education model was discussed with a focus on the recent shift in Germany from a more education led system with long years of study towards a more applied business oriented model with the introduction of a bachelor degree as a first qualifying degree. Family planning has been seen as shifting from very young parents to becoming parents at an older age, leading to a decrease in the overall fertility rate as families then have fewer children. Also, a shifting pattern in the female participation in the work force can be seen, with more women acquiring tertiary education (Muskat, 2008).
Within the case study the methodological structure used was to have a qualitative approach overall with data collection using expert interviews, having key terms determined and then displaying findings through building scenarios. Usually, the determination of key terms after transcribing interviews would be done with a content analysis (Krippendorf, 2004;Krippendorf and Bock 2008). However, within this research design a quantitative element was favoured in order to reduce bias in determining key terms and enable a more structured approach to determining the scenario options. A software supported cross-impact analysis was chosen to allow the computation of a larger amount of possible scenario data. Note. "qual" stands for qualitative, "quan" stands for quantitative, "+" stands for concurrent, "→" stands for sequential, capital letters denote high priority or weight, and lower case letters denote lower priority or weight.
Source: Johnson and Onwuegbuzie, 2004:22. Notation based on Morse, 1991. With reference to table 1 the design of this research was, therefore, to be found in the lower right quadrant with a "QUAL → quan" sequential approach. As another major qualitative step is applied with building scenarios for displaying the results, the model would evolve to be a "QUAL → quan → QUAL" pattern.
Data collection with qualitative step expert interviews 28 experts were interviewed using a semi-structured interview guide (Holstein and Gubrium, 1995;Mayring, 2002). The interviews, which typically lasted for 45 minutes, were recorded using a digital recorder with a subsequent transcription. The approach of an expert interview is a speciality within the semi-structured interview as the experts are determined deliberately (Abels and Behrens, 2005;Bogner et al., 2005;Meuser, 2001;Meuser and Nagel, 1991;Schnell, Hill and Esser, 1999). The

Electronic Journal of Business Research Methods Volume 10 Issue 1 2012
www.ejbrm.com 12 ©Academic Publishing International Ltd experts were either senior academics or senior managers in demography, geography, tourism, economics, human resources and statistics, who were able to discuss the particular scenario contexts. They contributed to at least one of the following fields, later called descriptors: politics, workforce, migration, fertility, education, market segments and travel behaviour. The experts had been chosen so that most could contribute to several fields, ensuring that at least 7 interviewees were contributing to any one of the descriptor topics.

Quantitative cross-impact-analysis using the software Szeno-Plan
Cross-impact analysis is a method for revising estimated probabilities of future events in terms of estimated interactions among those events (Dalkey, 1972:341;Turoff, 1972). The probability for a certain event to come about is affected by the occurrence or non-occurrence of another event. Crossimpact analysis permits the determination of the probability of two events occurring simultaneously. Using this combination of different descriptors leads to an array of new possible outcomes not thought of at the initial stage of data collection. It is this multitude of new combinations which will lead to scenarios both new and surprising for the researcher and the experts involved. Smith et al."s (2005) have presented a research sequence with the same components "interviewscross-impact analysis -scenario building", but remained solely on the qualitative side (Smith et al. 2005;Godet, 1987). We will see that the here presented mixed methods approach is a development from there and highly beneficial in its uniquely new approach. The benefits and the detailed development will be contrasted at a later point at the beginning of the part "Unforeseen results and credibility".
Cross-impact analysis can be used as a tool for evaluation leading to several ways of showing results, e.g. using it for simulation modelling (Fink, 2001;List, 2003;Tulbure, 2004;Brauers and Weber, 2006). In the featured case study, the topics consumer, travel and leisure behaviour and education were used as implications of demographic change, so that cross-impact analysis could be used in order to build scenarios based upon the results of the expert interviews. The information given by the interviewees was sorted by key words according to the different areas of expertise, field of interest or professional experience. According to the terminology used by Szeno-Plan these key words are called descriptors and comprised politics, workforce, migration, fertility, education, travel market and travel behaviour.
The descriptors condensed structure and defined the content of the expert interviews. They allowed the conversion of the large collection of qualitative data into an exhaustive set of variables that fed into the quantitative cross-impact analysis. When analysing the interview transcripts not only the descriptors themselves were identified, but at the same time descriptor tendencies were established. The descriptor tendencies indicated a personal tendency according to the following table 2.
Descriptors (e.g. politics) and descriptor tendencies (e.g. "being demographically aware" vs. "not being demographically aware") are shown here with additional information. Using the software Szeno-Plan is not itself novel: Lindgren and Bandhold have shown an example with 6 variables, i.e. the herein named descriptors (Lindgren and Bandhold, 2003:155). However when there are 6 variables a manual analysis of each interdependency is still feasible, generating a maximum of 64 (=2 6 ) results. With having n=2 for each descriptor tendency while having m=7 descriptors we can think of obtaining a maximum of n m =2 7 =128 different CIS in our case. As the output always doubles with each additional variable, the use of the software Szeno-Plan is recommended for cross-impact analyses with 7 or more variables.

Frequency and consistency
Two major outputs can be obtained from a quantitative cross-impact analysis namely listings according to frequency and to consistency.
"Frequency" here refers to how often any mathematically possible cross-impact scenario (CIS) output can and does occur (here 2 7 =128). In reality some of the 128 CIS permutations will be the same. As they occur more often, they are called "strong by frequency".
On the other hand the term "consistency" refers to different strengths in interrelationships between the descriptor tendencies. By filling in the level of dependence for each variable (i.e. descriptor tendency,  (Lindgren and Bandhold, 2003:155).
The example in table 3 reads as: The cross-impact scenario (CIS) with the running number 3 is determined to include demographically aware politics, modern workforce and social security driven migration. The CIS with the running number 10 is slightly different; the politics are not demographically aware, it has a traditional workforce, but again the migration is driven by social security reasons. A first result of the software Szeno-Plan at this stage was the frequency of all cross-impact scenarios. Szeno-Plan computed 128 different combinations overall, with some more frequent than others: for example, the two CIS used in table 3 occur very frequently with 10 CIS overall with the same combination as CIS no. 3 and 5 CIS overall with the same combination as CIS no. 10.
Szeno-Plan additionally interprets consistency when the input data for the cross-impact matrix is filled in according to a Likert scale from -2 to +2 with -2 stating that there is no influence at all and +2 for having a strong influence. The evidence for the several figures was to be found within the transcribed expert interviews. Each descriptor tendency was determined as having, or not having, an influence on each descriptor tendency. Tables 4 and 5 show excerpts of the overall table depicting the vice-versa influences and their interpretations. a) There is a strong influence (+2) of demographically aware politics on modern workforce. b) Politics which is not demographically aware has a strong negative effect on modern workforce (-2). c) Demographically aware politics does not seem to have an influence on a traditional workforce (0).
d) The same as in c) is true for non-demographically aware politics. Again there is no influence to be seen on a traditional workforce (0). a) Modern workforce has a weak positive effect (+1) on demographically aware politics. b) Traditional workforce has no effect (0) on demographically aware politics. c) Modern workforce has no effect (0) on non-demographically aware politics.

Building scenarios
Scenario technique as a qualitative method allows the description of possible complex situations in the future. It is used to integrate findings out of quantitative or qualitative analysis (Tulbure, 2004, p. 29). Historically developed within the military to display the outcome of different strategies, it has been further developed by the Frankfurt Battelle Institute in the 1980s and from there found its way into strategic corporate planning (List, 2003;Fink, 2001). With the rise of strategic and management consultancies in the 1990s it became a consulting and evaluating tool there (Fink et al., 2001;Gausemeier et al., 1996).
According to Fink (2001) scenario technique covers three steps:  Analysis on the scenario field finds descriptors and their cross linking.
 Within the step prognostic several basic alternatives are described.
 The final scenario development then checks on consistency between the several projections. The remaining 2 to 8 plausible scenarios can then be analysed, interpreted and described in a communicable form.
To display a typical scenario, chosen from several possible ones, more often than not a trend variant will be used. It is important to mention, however, that all depicted variants within a scenario cone (cp. fig. 2) are equally probable. A prediction of the probability of occurrence relative to the other scenarios is not possible. The trend variant is not the most probable but at best the least extreme.
trend variant future event today Source: Muskat, 2008:91;modified from Gausemeier et al., 1996:109. In the case study being discussed in this paper the output data of the cross-impact analysis was then presented in form of 4 scenarios, constructed out of the strongest 2 CIS by frequency and consistency respectively. The in depth scenarios, indicated as the most pertinent from the quantitative analysis, were then constructed using a qualitative approach based upon an in depth, thematic analysis of the expert interviews, whereby the specific expert knowledge was used to build scenarios according to the given descriptors. The specific scenarios are not shown within this paper which intends to only ISSN 1477-7029 give an overview of the method used, but more information on their content can be obtained from the corresponding author.

Evaluation of quantitative results
The outcomes of the quantitative step cross-impact analysis are CIS that are frequent and consistent at the same time. This allowed building scenarios with a much wider range of different descriptors than would have been possible with only a qualitative thematic analysis. Furthermore cross-impact scenario data had been presented with descriptor tendencies that was not in the focus of research and might have not been chosen with using a qualitative analysis. These emerging, less biased results were evidence of the higher level of quality pursued with the decision to use the mixed methods approach over a pure qualitative approach. The additional quantitative step opened up both a new horizon of possible cross-impact scenarios and a demonstration of which of these CIS were frequent and consistent. Assuming a typical qualitative research setting at this stage, we most probably would have seen a thematic text analysis with the necessary decision to be made concerning which data to use for building scenarios based upon the most interesting or most specific. In the current setting, however, we can use the quantitative analysis to show that the data used to underpin the scenarios are the most frequent and most consistent.
Another interesting aspect of the use of the mixed methods, demonstrating a possible improvement, is to be found in the area of utilising expertise, and how able the researcher is with the role of coordinating this expertise. Assuming again an overall qualitative setting, we find the researcher in the uncomfortable position of assuming the role of a decision maker, in terms of which of the provided facts within the interviews to select while not having equivalent expertise as any of the interviewees. Moreover, the interviewer is not an expert in all the fields, but neither are the experts: they only hold expertise in their specific research or business field and would not be experts in other areas. The system presented here enables the selection to be undertaken in a more structured way, utilising the expertise but guiding the researcher at the same time providing reliable data.

Unforeseen results and credibility
At this stage, contrasting the presented mixed method approach to a purely conducted qualitative approach is useful in shedding more light on the benefits of the former. Lindgren and Bandhold (2003) have presented cross-impact analysis both as qualitative and quantitative approaches and Smith et al. (2005), having used the same sequence of data gathering, analysis and presentation as the present paper, have opted to have their cross-impact analysis undertaken in a purely qualitative way.
The approach demonstrated within this paper has two clear potential benefits, when dealing with size or opportunity and when dealing with bias. Smith et al. have been using three three-day workshops with varying participants of 25 to 35 for each workshop (Smith et al., 2005:4). In some research situations it might not be advantageous or technically possible to have series of large workshops like these, and there are benefits of the sequence proposed in this paper over the traditional approach. We have already demonstrated earlier that with 7 or more variables (descriptors) the approach proposed in this paper is the feasible one.
Addressing the issue of possible bias we hear from Smith et al."s report that "the outcome of this exercise [i.e. the cross-impact analysis] was the identification of the main driving forces […] After some further discussion in a plenary session, it was agreed that there were two sets of variables […] from which the desired scenarios could be built." (Smith et al., 2005:7+8; our italics). In contrast this paper advocates the acceptance of cross-impact analysis outcomes as is, without the bias of several participants to have to come to an agreed majority decision. It is the current paper"s research advantage to have results unforeseen by the researchers involved who would not have had been inclined towards these specific factual outcomes.
Only through the use of a quantitative cross-impact analysis can a meta-level emerge that allows the researcher to see a new and surprising outcome. It is this specific data analysis step that combines the content of different expertise holders and presents a wide range of here 128 possible crossimpact analysis scenarios (CIS). Instead of bringing the experts together again in a second data collection step (cp. Delphi Method, e.g. Linstone and Turoff, 2002) and have them decide on convincing outcomes (Smith et al., 2005), we can confidently claim at this stage that, from the variety of CIS, there are some more frequent and more consistent than others. At no point within a solely www.ejbrm.com 18 ©Academic Publishing International Ltd qualitative approach would this claim be possible. We can conclude that a cross-impact analysis created a new angle with surprising results, consequently giving credibility at the same time.
Within the given case the outcome was that a highly consistent and frequent CIS would have the factors rise in fertility, a migration driven by social security and a travel behaviour with a cohort effect (cp . table 3). It is then possible for the researcher to go back to the transcriptions of the different interviews and find exactly the right passages that correspond with the CIS findings. Using these passages would result in the development of the final qualitative scenario building to present the findings of the research. These findings are less biased by the author as the composition was prescribed through the quantitative research step.

Critique and outcome
A potentially critical step in determining a qualitative result from the transcribed interviews is the finding of key terms, in this case called descriptors, within the text. Depending upon the method used, evidence can be found or overlooked or evaluated to different degrees. For the underlying work of the method presented in this paper the evaluation of which descriptors to find within the transcripts to be used as input data for the qualitative step was done by the author by reading through the interviews, highlighting passages and then allocating descriptors and descriptor tendencies manually.
In hindsight this is seen as a lack of methodological approach in determining the evidence for the descriptors. At the time of using Szeno-Plan for calculating the cross-impact probabilities it had not been discussed if the input-data for Szeno-Plan should have been determined in a quantitative way as well. Instead the data was found through qualitative text analysis, i.e. finding evidence within the interview for descriptors and their descriptor tendencies. It was then at this point decided along the interviewee"s expressions if the expert strongly disagrees with a combination, is indifferent or favours a given combination. It is recommended that in future studies, a researcher would use a text analysis tool, e.g. Leximancer in order to further justify choices.
Having decided upon a "QUAL → quan → QUAL" method it can be discussed how and if alterations within the methodological approach would have lead to different outcomes. The qualitative steps 1, the expert interviews, and 3, the scenarios, have been held fixed from the very beginning of the research planning, with the quantitative step 2, the cross-impact analysis, being the addition. Having decided upon a quantitative second step within an overall mixed method approach, there are several possible ways of combining input, analysis and output methods. In order to generate data for output scenarios for prospective forecasting studies cross-impact analysis is recommended, different research disciplines will use different software.

Conclusion
The advantage of the quantitative layer in the demonstrated case study can be seen when comparing with a purely qualitative approach. In that case a qualitative step 2 using a thematic text analysis or discussions in workshops would be used to generate the input data for the scenarios. But this has been seen as being prone to be biased by the researcher in favouring some topics unintentionally while omitting others. The strength and the value of this mixed methods approach is that it provides researchers with a broader set of analysis and a more substantial way of data interpretation. This paper demonstrated that qualitative data collection, combined with quantitative data analysis will lead to potentially more robust results. The outcomes of the research outlined within this paper could not have been anticipated with one method only, neither qualitative nor quantitative.
As this paper used qualitative expert interviews, followed by a quantitative cross-impact analysis we can further claim to offer and demonstrate results of high frequency and consistency. Those emerging less biased conclusions that can be drawn were additionally obtained and contributed well to the qualitative step. Instead of only having the accumulated expertise of all interviewees for consideration, a new level of results has become apparent. It is this added quality which contributes to the elevated value of mixed methods research. Thus, we recommend using a quantitative layer within a qualitative research to reduce bias. Mixed method research is beneficial to use in order to produce a research output that is of higher value than single approaches in qualitative or quantitative methods.