Can intuitive and analytical decision styles explain managers ' evaluation of information technology ?

Purpose The purpoi5e of this paper is to clarify how IT managers’ decision styles affect their evaluation of information technology. Design/methodology/approach Four different decision styles were assessed in a leadership test directed towards IT managers. Each style included two dimensions: confidence judgment ability and decision heuristic usage. Participants belonging to each style were interviewed and their answers analysed with regard to their reasoning about central areas of IT management. Findings Results suggest that a decision style combining intuitive and analytical capabilities lead to better evaluations of information technology. Originality/vale The results of the present study are valuable for the understanding of how decision styles impact on IT management in everyday life.


Introduction
There is at present a need for research that addresses the issue of how IT managers' decision style impact on their evaluation of information technology in everyday business life.Studies addressing this issue are scarce, and the understanding of IT management would benefit from insights provided by research in judgment and decision making.
Decision styles may be regarded as qualitatively different, generalizablc abilities that cut across task content (Hodgkinson et a l, 2008), and can be considered a personal trait in this respect.Executives have a default style of decision making developed in their careers, and that style is reinforced through repealed successes or changed after several failures (Williams and Miller, 2002).Decision styles control the ways or modes in which individuals perform particular tasks.They hereby differ from cognitive strategies, which are optimal procedures used to perform specific tasks (Beckman, 2002).
Intuitive decision styles may be more effective than rational styles to decision making in ill-structured managerial tasks, whereas an analytical decision style may be The authors would like to thank Ole Boe and Ingrid Lindstrom for assisting with the data collection and for helping with the coding of the protocols.more appropriate for well-defined managerial tasks (Dane and Pratt, 2007;McMackin and Slovic, 2000).Here we suggest that a manager's performance on classical heuristic tasks as well as on confidence judgment tasks together constitute a manager's decision style.Research on the implementation of information technology shows how managers often underestimate the complexity of such projects (Martins and Kambil, 1999).Partly as a result, many projects fail and end up becoming more costly and time-consuming than initially anticipated (Gottschalk, 1999), Decision styles influence managerial governance of IT, which in turn influences the returns on IT-investments.Yet, few managers can explain how IT is governed (Weill and Ross, 2004).Hence, it is vital for managers to clarify: • what decisions must be made in order to ensure appropriate management and use of IT; • who should make these decisions; and • how these decisions will be made and monitored.
The present study investigates to what extent IT-managers' decision making styles (intuitive or analytical) influences the managers' judgment of which decisions are regarded as central, who should make them, and how.

(k n em l outline o f the article
The article is organized as follows: we start by introducing the two central concepts followed by the hypotheses.Subsequently we introduce the design, give a brief account of the participants, present the procedure/material and highlight other methodological issues of importance.In the next section, the results are presented, and finally theoretical and managerial implications of our findings are highlighted.

Confidence judgments
Managers' ability for making realistic confidence judgments constitutes one of two major factors distinguishing intuitive from analytical decision processes.Levitt and March (1988) suggest that certain properties of interpreting an experience stem from features of individual inference and ju d ^e n t.People in general are not perfect statisticians and make systematic errors in recording the events of history and in making inferences from them.
Managers' ability to make correct confidence judgments of their own knowledge influences both their potential to make correct forecasts (Astebro and Koehler, 2007;Barber and Odean, 2000;Carhart, 1997) as well as their real-world decisions (Hayward et a i, 2006;Lovallo and Kahneman, 2003).For instance, it has been shovm that calibration accuracy of managers' judgmental processes is able to fairly well predict the commercial success of their new ideas (Astebro and Koehler, 2007).Overconfident managers are also more likely than well calibrated ones to allocate resources in their ventures (Hayward et al., 2006).More specifically, managers' propensity for overconfidence can be linked to their decisions to allocate, use, and attain resources.Overconfident managers tend to deprive their ventures of resources and resourcefulness and thereby increase the likelihood that their ventures fail.
Optimistic overconfidence constitutes a focal bias towards the desirable outcome and is often prevalent in business settings (Kahneman and Ix3vallo, 1993;Kahneman and Tversky, 1995).In addition, financial forecasts on the whole tend to be consistently

1327
Intuitive and analytical decision styles MD optimistic (Hogarth and Makridakis, 1981).Financial analysts on average predicted 4 g 9 that one of the most important stock exchange indices in the US would grow by 21.9 ' per cent per year from 1982 to 1997, whereas the actual growth averaged 7.6 per cent (Cadsby, 2000).Similarly, a survey of maaoeconomic predictions from 14 OECD countries found that industrial firms' production estimates were systematically over-optimistic in their production estimates (Madsen, 1994). 1328 Executives in organizations are found to routinely exaggerate the benefits and -------------------------------discount the costs in planning major initiatives ^ovallo and Kahneman, 2003).When pessimistic opinions are suppressed and optimistic rewarded, an organization's capacity for critical thinking is seriously undermined.This does not imply that optimism is bad, or that managers should try to root it out of themselves or their organizations.However, optimism and realism need to be balanced -between goals and forecasts, as suggested by Lovallo and Kahneman (2003).

Complex decision heuristics
Manager's ability to use complex decision heuristics correctly is another key factor that makes it possible to distinguish between intuitive and analytical decision processes.Koehler et al. (2002) have shown that much of the biases that can be attributed to managers' overconfidcnce can also be related to the improper use of decision heuristics.
The reason is that bavSe rate neglect appears to be crucial in both cases.In decision research, it is commonly assumed that the use of different decision heuristics is an adaptive response of an information processor with limited capacity for information processing (Baron, 1994;Payne et al., 1993).Such demands may include how complex the problem is and whether or not uncertainty is involved.One reason for using decision heurivstics in everyday life is that optimal strategies often are unknown or unknowable (Simon, 1987).However, the term heuristics may not only be understood as equivalent to rule of thumb.For instance, Baron (1998) argues that the term intuitions may, in many cases, be as relevant here since the principles in question are not always used as rules of thumb.Sometimes the use of complex decision heuristics may lead to characteristic errors or biases (Kahneman et al., 1982;Tversky and Kahneman, 1974;Gilovich et al., 2002).It has therefore been argued that the focus of the researcher, in this context, must be on the features of the complex decision heuristics that may lead to judgmental or perceptual biases (Kahneman and Tversky, 1996).
In the present study, we examine how the use of complex decision heuristics introduced by Tversky and Kahneman (1974) may influence information systems managers' reasoning with regard to strategic issues in their organizations.Thus, we focus on traditional complex decision heuristics such as the availability heuristic, the represenativencss heuristic, and the anchoring and adjustment heuristic.It is assumed that the impact of these complex decision heuristics on real life decisions wdll result in different information processing behaviours, based on the fact that all these complex decision heuristics basically reflect peoples' inability to interpret information in a logical and statistically correct way.It has recently been shown that IT managers frequently use decision heuristics in a way that results in biases (Fuglseth and Gronhaug, 2003).However, we believe that it may be less easy to actually trace the usage of complex decision heuristics to the perception and evaluation of IT importance.
PrevioUvS research has revealed that high performers' decision processes compared to low performers', are less influenced by the framing of information (Selart et a l, 2006).
High performers search for more information and spend more time on the process.However, performance on confidence judgment tasks seems to have a greater impact than the execution of the decision heuristic tasks in these respects.Hence the following hypothesis can be made: H I. Overconfidence will have a greater impact than the use of complex decision heuristics on the way IT managers perceive and evaluate the importance of IT.Overconfidence will kx)m larger than complex decision heuristics as a factor when differences due to managerial decision style are detected.
H2.The decision style of an IT manager will be able to predict how he or she perceives and evaluates the importance of IT.

1329
Intuitive and analytical decision styles

Design
In order to test the stated hypotheses we apply a design in which a cognitive leadership test is used assessing two central dimensions of leadership cognition, namely decision heuristic usage and confidence judgment ability.In this connection, three different decision heuristics are investigated (representativeness, availability and anchoring/adjustment).Based on the test, four managerial decision styles are devek)ped, built on high versus low performance on each dimension.In addition to the test, interviews are held with each manager.These interviews focus on four central areas of IT management, namely: (1) visions regarding IT in the organization; (2) motives for IT development; (3) opportunities and obstacles to the development of IT; and (4) measurement of IT.

We test our hypotheses by relating the four different decision styles to the answer patterns tied to the central areas of IT management in the interviews.
Participants A total of 27 information system managers participated in the study and were recruited from organizations with over 130 employees.The mean number of employees was 4,567 (the size of the organizations varied between 130 and 55,000 employees).The majority of the organizations were dealing with manufacturing and service (both public and private).
The mean age of participants was 42.8 years (SD = 10.2) within a range of 21 to 60 years.The managers differed with regard to their background, level of education, and their position in the organization.They also differed with regard to the type of organization they worked in, how long they had been working with information systems, and for how many years they had been working for the organization.In total, 18 participants were in charge of their company's respective division for information systems.Ten participants were also responsible for the economy and the administration in addition to being responsible for the information systems, or they were responsible for the whole business.Taken together, four participants were members of the executive board, 11 participants were part of a managerial group, and twelve participants had positions below the managerial group level.When looking at levels of education, it turned out that 16 participants had some form of higher education whereas eleven had some form of high school diploma as their highest education.

Materials and procedure
Participants were contacted by telephone and were asked to participate in the investigation that consisted of an interview and a second part where they would have to make judgments with a relevance to decision theory.They were told that the aim of the study was to collect data for a scientific study.The interviews were in most cases performed at the participants' work places and were conducted m a semi-structured form.The interview followed the same format with all of the participants.First, participants were requested to provide some background data.The interview was then conducted and recorded.
A brief questionnaire was distributed to the pari:idpants prior to taking part in interviews.The questionnaire contained some general questions about background variables, such as age, gender, level of education, and previous employment.Additional questions were subsequently asked about how many years participants had been working in their profession and in their organization, the type of organization they worked in, as well as their position in the organization.
The interview questions mainly concerned visions, current strategies, evaluations etc. related to the management of information systems and were based on the results obtained from previous research (Allwood and Hedelin, 1996; Hirscheim, 1989;Mayo, 1991;Weill and Ross, 2004;Van Bon et oL, 2005).The focus has been on IT governance, that is, on the accountability frameworks that exist in order to encourage desirable behaviour in the use of IT among managers.The questions that we applied were aimed at investigating the use and development of information systems in the organizations.They were designed such that they would address relevant managerial issues as well as providing ecologically valid data from everyday life pertinent to the hypotheses.The relevance of the questions to management had been secured in a pilot study.In the interview, participants were presented with a set of vsemivstructured questions that are presented below: (1) Visions regarding IT in the organization: • What kind of visions do you have regarding the use of IT in your organi7^tion?
• How will your visions affect the processing of information in the decision making process?
• How sure arc you that these visions are the correct ones ?
(2) Motives for IT devebpment • What are your motives for the IT development?
• What can you gain from IT development in the future?
• How do your motives affect the processing of information in the decision making process?
• How sure are you that there are not any other important motives?
(3) Opportunities and obstacles to the development o f IT: • What are the opportunities and obstructions for effective use of IT in your organization?
• How do the opportunities and obstructions affect the pnx;essing of information in your decision making?-------------------------------The heuristic tasks and the confidence judgment tasks were related to theory in the sense that they made it possible to create four decisions styles based on high or low performance on each of the two types of tasks.
After having completed these tasks, participants were informed about the full purpose of the study.A full session including the interview and the decision heuristics tasks lasted between one-and one-and-half hours.

Coding oftlie protocols
The data from the interviews were coded into different categories according to content and the question asked.This was achieved by the creation of a coding scheme that was subject to consensus by several independent coders with regard to the relevance of the categories for the hypotheses.In each interview protocol a number of key paragraphs were identified as meaningful narratives of the interview.Subsequently, these narratives were coded with regard to the established coding scheme and the coding subjected to an inter-coder reliability test.Intercoder reliability is a measure of agreement among multiple coders for how they apply codes to text data and can be used as a proxy for the validity of constructs that emerge from the data.In order to test the reliability of the category codings we applied the Cohen's kappa coefficient which is a statistical measure of intercoder reliability.It is generally thought to be a more robust measure than simple percent agreement calculation since k takes into account the agreement occurring by chance.Cohen's kappa measures the agreement between two raters who each classify N items into C mutually exclusive categories.If the raters are in complete agreement then k = 1.If there is no agreement among the raters (other than what would be expected by chance) then k < 0. Intercoder reliablity of our category codings summoned up to k = 0.61.

Construction o f task measure
The heuristic task measure was constructed by summing the the number of times the respondents answered correctly across the 24 different heuristic tasks.A correct answer was always coded as 1, and an incorrect one as 0. If participants chose the correct answer, the corresponding confidence rating was given a positive value, otherwise it was given a negative value.An index measure of confidence was obtained by taking the mean values of the confidence ratings of the same 24 tasks.All participants that performed above the mean value on the choices (M = 10.77,SD = 1.77) and on the confidence ratings (M = 77.14,SD = 7.50) were coded as high achievers, and those performing below or equal to the mean values were coded as low achievers.In this way, it was possible to create four groups of participants; one group consisting of high achievers with high accuracy and calibration (logical-statistical and balanced confidence), and a second group of low achievers with low accuracy and calibration (biased and overconfident).Furthermore a third group was constructed consisting of those participants that performed below or equal to the mean value on the choices but above the mean value of the confidence ratings (biased and balanced confidence).Finally, a fourth group consisted of those participants that had performed above the mean value on the choices but below the mean value of the confidence ratings Qogical-statistical and overconfident).

Results
This section reports the findings from our investigation, focusing on the impact of the four managerial decision styles on: (1) visions regarding IT in the organization; (2) motives for IT development; (3) opportunities and obstacles to the development of IT, and (4) measurement of IT

Decision style impact on m anagers' descriptions o f visions regarding I T in the organization
Chi-square tests were performed on each of the four decision heuristic group.The means for all groups with regard to managers' descriptions of visions regarding IT in the organization are presented in Figure 1.
The answers were coded into three categories (see Figure 1 The tests revealed that for the logical-statistical and balanced confidence group and for the biased and overconfident group, no significant diffrences were revealed between the answer categoricvs, X\ = 2.00, n.s., and x ( = 2.00, n.s., respectively.For the biased and balanced confidence group a significant difference was observed, was yielded by x\ = 4.00, p < 0.05, indicating that these managers stated reliably more Category 1 than Category 2 answers.For the logical-statistical and overconfident group a significant difference between the answer categories was also sustained, = 4.50, p < 0.05, revealing that also participants in this group gave significantly more answers belonging to Category 1 than to Category 2. A chi-square test performed on the combined four groups yielded a significant difference between the answer categories, as sustained by = 9.85, p < 0.01.This effect again indicated that significantly more Category 1 than Category 2 answers were stated.

1337
Intuitive and analytical decision styles

Discussion
It is generally agreed upon that decision makers in organizations should act logical, statistical, and with a balanced confidence in their own knowledge in order to achieve success.Biased perceptions and overconfidence on the other hand are most often associated with failure, in the literature (Baron, 1994(Baron, , 1998;;Kahneman et al, 1982;Kahneman and Tversky, 1996).
In the present study, we investigated if confidcnce judgments and complex decision heuristics, treated as four distinctive leadership styles (logical-statistical and balanced confident, biased and balanced confidence, biased and overconfident, and logical-statistical and overconfident) were able to explain managers' issue interpretation wath regard to how they regarded the role of IT in their organization.
Overconfident managers saw information technology more as innovative resources for company activities than as a vehicle for the development of organizational structures and routines or technology substitution.An emphasis on visions as innovative resources rather than as vehicles for promoting efficiency, may suggest that these managers were victims of a control illusion.The managers' control illusion in turn may be connected to a higher willingness to invest resources in different types of ventures (see also Astebro and Koehler, 2007;Hayward et at., 2006).Unfortunately, necessary data for further analysis were not available.
Executives in organizations routinely exaggerated the benefits and discounted the costs in planning major initiatives (Lovallo and Kahneman, 2003), effectively undermining critical thinking in the organizations.Optimism does not necessarily undermine effective decision making in organizations but managers need to strike a balance between optimism and realism -between goals and forecasts, as suggested by Ivovallo and Kahneman, 2003.

Implications o f the results
The most "rational" group Oogical statistical and balanced confidence) tended to interpret the issues so that no reliable differences could be found between the answer categories.This suggests greater flexibility with respect to perspective taking.
For the other styles, reliable differences were sometimes observed among members' different views (see also Allwood and Hedelin, 1996;Hirscheim, 1989;Mayo, 1991; MD Ward et a l, 1996).Hence, the results of the study suggest that rational managers, as a ¿10 g i^oup, predominantly engage in different strategic issue interpretations as a way of ' risk reduction.It is difficult to spot a clear consensus within this group about one "correct" interpretation when different issues are at stake.Thus, the group seems to favour risk aversive rather than risk seeking reasoning.Furthermore, overconfident and decision biased managers differed from all the 1338 other groups in their perspective on the factors obstructing the development of IT. -------------------------------Overconfident and biased managers saw inadequate competence and centralization as more important compared to costs, inadequate management, and infrastructure.Reliable differences were not observed for the other groups.Managers that performed low on both the classical heuristic tasks as well as on the confidence judgment tasks pinpointed factors as more important that to a lower degree were linked to their own responsibility.Instead, the factors that were addressed as most important appeared to have clearer connections to others such as top management and employees at large in the organization.It is important to note that inadequacies due to heuristic reasoning as well as the overconfidence may contribute to this effect.The general picture is that low performance on both tasks contributes to a focus on external rather on internal issues and differences in k)cus of amtrol and/or accountability due to this may be assumed among the managers (e.g.Selart, 1996Selart, , 2005)).A possible explanation can be found in base rate neglect of own justifiable actions that leads to collective successes and failures.Interestingly, the results suggest that intuitive judgments may be more diagnostic than rational approaches to decision making in ill-structured judgmental tasks (see also, Dane and Pratt, 2007;McMackin and Slovic, 2000).In sum, the findings suggest that overconfidence plays a larger part in explaining inadequate evaluations of IT-systems than biased decision making.Overconfidence may compared to biases be more inhibitive on evaluative activity.Lipshitz (1995) points out that the uncertainty associated with real-world decisions interrupts ongoing action, delays intended action, and guides the development of new alternatives.Intuitive reasoning is not by itself more effective than analytical approaches in ill-structured tasks (see Dane and Pratt, 2007;McMackin and Slovic, 2000).Rather, success is likely to stem from a mixtiire of intuition and analytical capabilities.In such a mixture of capabilities, balanced and calibrated confidence judgments seem to be of key importance.The realism of these judgments appears to be supported by the use of fast and frugal decision heuristics, that is, both analytical and intuitive processes give the impression to support calibrated confidence judgments (Gigerenzer et al., 1999).It is not established that the interplay between analytical and intuitive capabilities has the same successful impact on the use of complex decision heuristics.

Limitations
The number of managers interviewed for the study was relatively small due to practical reasons.The industrial sites were geographically spread over a mid-sized county area and had to be reached by car.Hence, each interview required a couple of hours for the realization and recording alone.Transcription and coding of each interview protocol would add an additional two hours.Neither did the study include measures on managerial performance that is to what extent overconfident or biased managers, differed from the others, in terms of efficiency and performance.

Future research
In the paper we looked at how overconfidencc and complex decision heuristics influence managers evaluation of information technology.New research suggests that a meaningful distinction can be made between complex and simple decision heuristics (Gigerenzer et a l, 1999).Simple decision heuristics are fast and frugal and often outperform more formal, statistically informed reasoning.Simple heuristics perform well and produce accurate forecasts in difficult, real life situations (Astebro and Elhcdhli, 2006).While simple heuristics have been studied in a series of different contexts, few studies have looked at the effects of simple heuristics on how managers evaluate information technology.This, we suggest represent a promising direction for future research.

1339
Intuitive and analytical decision styles

( 4 )
Measurement o f IT effects:• How do you evaluate the effects of IT in your company?•How sure are you that that there are not any other important effects?Following the interview, participants completed a series of tasks to measure their heuristic making style.The heuristic decision making style was assessed by decision heuristic tasks as well as confidence tasks (Selart et al., category consisted of six questions, and the four decision ' heuristic categories included were representativeness, availabihty, and anchoring/adjustment.Each question always had a correct answer and a wrong answer.After having given an answer, participants were also requested to give a confidence rating on a half-ranged scale ranging from 50 (making a guess) to 100 1332 (absolute sure) to indicate how sure they were that they had given the correct answer.
Figure 1.Numbers of managers in respective groups and distribution of answers regarding IT visions

Further
Figure 3.Number of managers in respective groups and distribution of answers related to factors obstructing the development of IT