Business Intelligence Effectiveness and Corporate Performance Management: An Empirical Analysis

Business intelligence (BI) technologies have received much attention from both academics and practitioners, and the emerging field of business analytics (BA) is beginning to generate academic research. However, the impact of BI and the relative importance of BA on corporate performance management (CPM) have not yet been investigated. To address this gap, we modeled a CPM framework based on the Integrative model of IT business value and on information processing theory. Data were collected from a global survey of senior managers in 337 companies. Findings suggest that the more effective the BI implementation, the more effective the CPM-related planning and analytic practices. BI effectiveness is strongly related to BA, planning and to measurement. In contrast, BA effectiveness is strongly related to planning but less so to measurement. The study suggests that although both BI and BA contribute to corporate management practices, the information needs are different based on the level of uncertainty versus ambiguity characteristic of the management practice.


Introduction
Making strategic decisions in a dynamic business environment is a challenge faced by many organizations today. Although most organizations perform well in applying management systems in the areas of budgeting, financial and management reporting, and business intelligence (BI) analysis, the use of such systems for corporate-level decision-making is not as prominent. A corporate performance management (CPM) system is a tool that can help organizations address this challenge. These systems combine management practices and information technology (IT) to enable organizational performance [17,48]. Corporate-level management practices determine organizational success because they set the organization's strategy and enable its execution [48]. For example, case studies of Nortel, Circuit City, and Kodak [35,40,45] show that poor information processing and analysis, particularly with respect to corporate planning, played a significant role in the failure of these former Fortune 500 companies. BI technologies are thought to support CPM [36], yet, while research suggests that BI can help improve the effectiveness of operational processes [2,16,22], and its impact on management practices has been less studied [58].
Although organizations have used data to make informed decisions since the early 1900s [52], in today's organizations, the volume of data available has led to the creation of advanced analytics functions (which we will refer to as Business Analytics or BA). These practices involve approaches such as data mining that look for patterns not discernible using standard BI tools. Despite the recent shift toward the term Business Intelligence and Analytics [e.g., 8], traditional BI systems often deliver aggregated data, while the statistical methods employed in BA tend to use raw data. Furthermore, it has been argued that organizational needs related to analytic information differ from those related to more transactional information [33,50]. Therefore, differences exist between these two analytic approaches which suggest that a BI system that extracts and transforms transactional information might not necessarily support BA practices.
From the perspective of corporate performance management, the key practices include planning, measurement, and analysis [48,53]. While, as discussed above, the BI system might not necessarily support BA due to the different types of information and technologies used, we would expect that the BI system would strongly support planning and measurement. Yet, not all BI systems deliver the capability needed by decision makers [25]. Therefore, exploring the relationship between BI system effectiveness and the effectiveness of corporate-level management practices would provide an indication of the importance of these systems for CPM.
Finally, the emerging practice of BA could be considered a management practice that relies on the BI system but also influences the other management practices of planning and measurement. Since BA typically relies on raw data for statistical modeling purposes, however, we would expect that BA and the BI system influence planning and measurement in different ways.
Based on the above reasoning, our research questions are as follows: (1) What is the relationship between the effectiveness of BI system implementation and the effectiveness of corporate-level management practices? (2) To what degree does the BI system influence BA effectiveness in organizations? (3) What is the relative importance of the BI system versus BA for corporate-level management practices?
Addressing these questions would advance the BI value literature by examining the impact of BI on corporate-level management practices. In addition, assessing the relative impact of BA within CPM adds to the growing literature on the use of BA in organizations. From a theoretical perspective, we extend empirical work on the Integrated Model of IT Business Value to the corporate management domain and introduce Information Processing Theory and the concept of media richness as elements that are important in theorizing about information needs for management practices.
The remainder of this article has been structured as follows. The next section reviews the literature. We then outline the research method and discuss the findings. In subsequent sections, the implications for theory and practice are discussed, followed by the inherent limitations of our work and avenues for future research.

Literature review
Various definitions of BI and BA exist [3,38,55,59]. Conceptually, however, the delivery of data to decision makers involves technology (i.e., a BI system), and the application of data to derive insights requires the use of analytic techniques which can range from simple tables to more sophisticated statistical analyses. Our perspective in this study is that BA as a specific practice consists of these advanced statistical techniques that leverage data delivered through a variety of sources including standard BI systems [46].
The Integrative Model of IT Value provides a framework to guide empirical exploration of the role of technology in organizations [34] and thus serves as a basis for this study. This model does not, however, address the information needs of decision makers, an issue important for corporate management practices. We therefore also rely on Information Processing Theory which is a useful model for exploring the information needs of users. Based on the foregoing, in this section, we outline a theoretical foundation for the study that articulates our view of BI and BA as well as our focus on implementation effectiveness as a key construct, we also provide a definition of "management practices" and describe the research model.

Theoretical foundations: BI and BA
Notwithstanding the various definitions of BI [3,38,56], BI systems are generally considered to be software applications that deliver information to decision makers to help maintain business performance [20]. Most BI systems include different technological components [2,56] such as databases, visualization tools, and on-line analytic processing (OLAP) that allow decision makers to view and work with subsets of data [8]. Arguably, visualizations, or OLAP could be considered a form of analytics, but the term Business Analytics typically refers to the use of statistical, data mining, or other quantitative methods to derive insights from data [14,29].
One of the critical questions addressed in the literature is the degree to which BI systems and BA deliver business value to organizations. Empirical evidence from several authors demonstrates the impact of BI systems on the operational processes of sales, marketing, and inventory management [2,15,16]. Aruldoss et al. [2] suggest, for example, that for sales and marketing, the BI system provides information on customer needs across many different segments thus helping managers to better match products to market demand. Moreover, because many of these decisions are considered to be "programmable," business rules are sometimes built into the BI system [16,27] to speed up the decision-making process. From a management perspective, these types of programmable decision-making situations fall under the category of "management control" that requires an exchange of information from operational levels to responsible managers and vice versa [32].
No such evidence is available, however, for the impact of BI on corporate-level management practices. One might conjecture that the BI system serves a similar role as for operational processes: that is, it provides historical information and permits an exchange of information among managers. The Integrative Model of IT Business Value [34] (Integrative Model hereafter) is a useful framework for understanding the role that a BI system plays in organizations. This model, based on the Resource Based Theory (RBT), suggests that technology influences business processes which in turn influence organizational performance. The foundation of the RBT is that resources of a firm provide capabilities that enable it to compete in the marketplace. Accordingly, the Integrative Model suggests that technology is an internal resource that must be accompanied by complementary assets, such as workplace practices, to enable business processes that lead to organizational success. Based on this model, corporate management practices could be viewed as strategic-level processes enabled by the BI system. Melville et al. [34], however, point out that one of the weaknesses of the RBT is that it assumes that all resources are used to their full potential. Their model further suggests that the way in which technology is deployed depends on the capabilities of the organization's information technology personnel. This notion is supported by research demonstrating that information technology capability does influence the quality of the systems adopted by the organization [5,6,30]. It is also clear that not all organizations derive value from BI systems because of mismatches between the capability required and that delivered by the system [19,25]. The implication is that organizations possess different levels of information technology capabilities, and that these capabilities likely influence the effectiveness to which a BI system is implemented.
This question of BI implementation effectiveness has been addressed in the BI maturity literature. Maturity, however, implies a progression in capability in which an organization evolves its BI system from fragmented applications to a full enterprise view [24,29]. In contrast, implementation effectiveness may be viewed as the degree to which system implementation meets organizational expectations: whether the system is doing what it was intended to do [20,47]. As emphasized in the BI maturity literature, a well-implemented system would provide easy access to high-quality information [38] that would enable corporate management practices. On the other hand, systems that were not as well implemented would have less of an impact on the effectiveness of these practices.

Corporate management practices
Corporate management practices are habitual activities designed to establish and implement the organization's strategy [47]. A practice is defined as a "habitual or customary operation" 1 : a series of activities done the same way each time the practice is implemented. A method is a particular form of procedure. Research demonstrates that organizations use a wide range of different tools and techniques [41,49] integrated into higher-level management practices. The practice of management accounting, for example, includes methods such as Activity-Based Costing or Balanced Scorecards, each implemented following a set of specific procedures [9]. Similarly, a corporate-level management practice such as planning would involve methods that include environmental scanning, SWOT (strengths, weaknesses, opportunities, and threats) analysis, or strategy mapping. For the purposes of this study, we define these practices as corporate-level customary operations that include a variety of different methods.
The key management practices involved include planning, measurement, and analysis [47,53]. These tend to be more complex than the programmable operational processes discussed earlier. Planning, for example, requires the integration of external and internal information from a wide variety of sources to chart a course for the company. It has been argued, in fact, that planning is primarily an information gathering exercise [44]. The practice of measurement involves defining performance indicators, gathering and analyzing data on current performance versus expectations and then taking corrective action as required [32]. Analysis makes use of a variety of different techniques that help decision makers integrate information in a meaningful way. For example, a visualization might define a trend line allowing managers to anticipate what might happen in the near future. Alternatively, sophisticated regression models might be used to provide more certainty about the trend line. Therefore, these more complex practices call for different types of information delivery and different analyses than do operational processes.
While the Integrative Model discussed earlier is useful for exploring the overall impact of technology in organizations, it does not address the information needs of specific processes or practices. To explore this question, Information Processing Theory (IPT) has recently been invoked [7,28]. This theory argues that organizations strive to fit information capability to information requirements. Complex processes characterized by high task uncertainty generate higher information needs. When uncertainty persists despite the delivery of additional information, buffers are often used. For example, in the case of inventory management, firms might hold "safety stocks" when managers are unsure about the level of demand for their products [39].
Equivocality, defined as ". . .the multiplicity of meaning conveyed by information about organizational activities" [12], is another important element of IPT. In this situation, information is available, but considerable ambiguity exists such that decision makers are not confident about what course of action to follow. Ambiguity is often present in non-routine, complex tasks and can be resolved through the selection of "rich" media. Media richness is defined as the information carrying capacity of an information medium [11]. The richest medium is face-to-face communication, and therefore, in ambiguous situations, face-toface encounters can help to build a shared understanding of the problem. Empirical support for this theory is provided by Kowalczyk and Buxmann [28] who demonstrate that companies facing complex decisions use group meetings more often during the decision-making process. In contrast, decisions that were considered routine relied exclusively on the BI system.
Media richness theory was developed in the late 1980s. Considering modern analytic capabilities in organizations, ambiguity reduction can be viewed from a different perspective. The BI system, for example, might serve to reduce uncertainty by delivering large amounts of information. On the other hand, methods used in BA, such as data-mining, could reduce ambiguity by sorting through large volumes of data to discover meaningful patterns. Therefore, while a BI system might be directly useful for practices characterized by low complexity, BA might be of more importance for situations featuring more complex and thus ambiguous decision-making conditions.

The research model
In summary, based on the Integrative Model and IPT, we argue that BI system implementation effectiveness influences corporate management practices which in turn has an impact on organizational process effectiveness. The practice of BA also directly influences planning and measurement. Figure 1 outlines the research model based on the above argument. In the ellipse at the bottom of this figure, consistent with the Integrative Model, we identify organizational process effectiveness as the key outcome. The other ellipses identify the relationships between and among the constructs in the study. The corporate management practices of planning and measurement are thought to directly influence process effectiveness because they represent different forms of control mechanisms [48]. Planning can be considered a "feedforward" mechanism in that it helps to ensure that employees conduct activities that accomplish business objectives. Measurement is a feedback mechanism to permit ongoing course correction. Both control mechanisms work in tandem [57] to enable organizational success.
BI system effectiveness (BIS Effectiveness in Figure 1) and business analytics (BA Effectiveness) are both modeled as influencing planning and measurement through the delivery of information. The BI system is also assumed to directly influence BA Effectiveness. The rectangles in Figure 1 depict the methods involved in each of the practices and the specific 1 Similar definitions can be found in any of the major English language dictionaries. technologies and techniques involved in the BI system and in BA. These will be discussed in more detail in the following section.

Data collection
This study was conducted in collaboration with two industry partners, PricewaterhouseCoopers (PwC) and the Canadian Advanced Technology Association (CATA). These industry partners collectively represent more than 50 years of experience in the field of performance management, and therefore, they confirmed face and content validity of the survey used in collecting data. Specifically, the partners had worked with many of the organizations involved in the study and therefore helped to identify the specific techniques involved in the CPM practices. The study used an online survey method for data collection. Survey questions were developed based on the research hypotheses and on feedback from the industry partners. The questions went through three rounds of reviews by a sub-set of respondents in the partner organizations. Survey respondents were recruited through e-mail invitations distributed to 1,300 senior managers from PwC's and CATA's databases. A total of 337 complete responses were received and analyzed using the partial least squares (PLS) with the SmartPlS 3 software [42]. PLS-SEM was selected because of the exploratory nature of the study.
Given our focus on CPM, we targeted senior managers, executives, and board members of the companies surveyed. The intent was to elicit the assessment of the most senior level managers who would have a corporate viewpoint of BI system implementation and CPM practice effectiveness. It has been argued that these executives can accurately perceive the value of IT investments through distributed sense making [51] because they often approve the BI system expenditures, and they are closely involved in many of the CPM practices. Moreover, their assessments tend to be accurate when compared with objective measures of the same phenomena [16], and the best way to assess user perceptions of system effectiveness is to ask them [25].
Since the informants provided responses to the dependent and independent variables, the marker variable statistical test was adopted to estimate the impact of Common Method Variance (CMV). Following Lindell and Whitney [31], a scale measuring the quality of information provided to the board of directors (Cronbach's alpha of 0.85) was included and used as a "marker" variable since it was thought to be theoretically unrelated to the dependent variable. This variable was comprised of four measures: information related to customers, employees, innovation and the impact of company activities on the environment. The PLS algorithm demonstrated high validity of the construct (t-values of the measurement variables ranged from 38 to 128 and factor loadings from 0.96 to 0.98). The correlation of the marker variable to other constructs in the study ranged from 0.02 to 0.08 with t-values ranging from 0.15 to 0.70. The largest correlation accounted for only 0.006 of one of the constructs (measurement effectiveness), which led us to conclude the CMV is not a problem in this study.

Constructs and measures
The logic of a PLS-based study is that unobservable constructs can be measured by gathering information on observable items related to that the construct. In this study, the central construct is the effectiveness of the BI system and CPM practices as measured by the perceived effectiveness of related components and methods (the observable measures in this case). We relied A considerable debate has emerged about the use of reflective and formative constructs in PLS-based research [1,10,14,26,32,37,43,54]. One area of consensus emerging from this debate is that constructs could be modeled either as formative or as reflective based on theory or on the objectives of the research. Jarvis et al. [26] defined four basic criteria for determining whether constructs should be modeled as formative or reflective. The first criterion refers to the direction of causality, does the construct determine the measures or do the measures determine the construct? Coltman et al. [10] suggest that whether the construct exists independently of the measures is also important and Petter et al. [37] argue that one should also consider whether all measures reflect a common theme. If so, the construct would be perceived as unidimensional and therefore reflective. The second criterion is the degree to which measures are interchangeable such that the construct does not change as the measures change. The third is the degree to which the measures are correlated, and the fourth considers whether the measures have similar antecedents and consequences.
In this study, respondents assessed the effectiveness of specific components of the BI system and the methods used in the three management practices to form an aggregate evaluation of overall effectiveness. We will apply the four criteria to assess the BI system effectiveness construct as representative of all constructs in the study. Conceptually speaking, BI system effectiveness (defined as the degree to which the system delivers what it is supposed to deliver) could exist without the existence of the BI system itself. For example, during the planning stages, managers would consider the BI capability needed [25]. This capability requirement defines in fact, the "effectiveness" needed from the BI system, and specific BI system components would be selected and implemented based on the planned requirements. The effectiveness of each of these specific components provides an overall measure of effectiveness of the BI system.
In terms of the second criterion, these specific components are interchangeable because our interest was on the aggregate notion of BI system effectiveness, not on any of the specific components themselves. Accordingly, the measures carry a common theme (i.e., does each component do what it was meant to do) that provides for an assessment of the overall system.
For the third criterion, it is difficult to know a priori if the measures are related, but one assumes that if an organization effectively implements one BI system component, the others will also be implemented effectively. Finally, with respect to the fourth criterion, each component would have similar antecedents and consequences: the BI system is implemented to solve information delivery issues and results in improved access to information. The consequence of a well implemented system is that decision makers have access to high-quality information [38]. This reasoning holds for all components of a BI system, and thus, the measures have similar antecedents and consequences.
The survey questions focused on BI system implementation and management practice effectiveness. In this case, effectiveness referred to the degree to which the BI system and the management practices achieved the intended goals [20,47]. Respondents were asked to record their perceptions of effectiveness on a scale of 1 to 7 with 7 meaning "highly effective." The overall score for effectiveness for each construct was the mean of the associated components and methods.

Validity and reliability
Convergent validity is confirmed when measurement items load more highly on their latent constructs than on any other construct and show a significant t-value [18]. Tables 1 and 2 provide the factor loadings for all constructs showing that their measures do in fact load significantly (p ≤ 0.01).
Discriminant validity is demonstrated when the average variance extracted (AVE) related to the latent construct is at least 0.50 and when the square root of the AVE is larger than the correlation of the construct with any other construct. Table 3 confirms that this is indeed the case.

Control variables
Research suggests that organizational size [23] and industry sector [13,16] can influence the ways in which CPM is employed in organizations. Accordingly, size and industry sector are considered as control variables for this study. For organization size, we use the number of employees as the basis for creating categories of small (less than 250 employees, 139 cases) and large (250 or more employees, 198 cases) organizations. For the sector variable, we collapsed the various industry sectors represented into service (200 cases) and nonservice (137 cases) based on the argument that firms differ most markedly in their use of CPM systems along this line of delineation [16].
In addition, it is possible that companies in Asia use BI systems differently than do North American companies [60]. Accordingly, we also collapsed the sample into East (China and Japan, 88 cases) and West (North America, 225 cases). Companies that did not fit into these categories were eliminated from the data set. Table 4 displays demographic information on the respondents and their organizations. Figure 2 provides the results of the PLS analysis (t-values in brackets). In Figure 2, we note a strong relationship between BI system implementation effectiveness and the effectiveness of BA. This finding indicates that the BI system might help to organize data for use in BA. The R 2 is 0.336, however, indicating that BI system effectiveness accounts for a moderate portion of the variance in BA effectiveness. We consider two factors to explain this result. First, BI system use is voluntary: decision makers could access information from different sources [4]. Second, in many BI systems, data are aggregated, but advanced analytics typically requires the use of raw data. Accordingly, as more advanced analytics techniques are adopted in organizations, the use of BI system-based aggregated data might decline in favor of storage systems (such as data lakes) that maintain data in more granular formats. Table 5 displays effect sizes for the relationships in the research model. BI system effectiveness has a strong influence on all management practices. The impact of BA is not as pronounced, and its effect size is not significant for measurement but it is for planning. This finding is consistent with information processing theory because, as previously discussed, planning could be considered a more ambiguous management practice than measurement. Accordingly, the application of more sophisticated analytic models might be used more so than in measurement.

Research results
We can see that measurement has a strong relationship (0.461) with process effectiveness, while planning has a weaker (0.375) but still significant relationship. These findings could be explained through control theory [32]. Planning is a feedforward mechanism and at corporate levels in the  organization is often done once a year. Measurement by contrast is a feedback device that is used on an ongoing basis to help make mid-course corrections to organizational activities. Considering Tables 5 and 6, we conclude that BI system effectiveness strongly influences BA and has a weaker but significant relationship with the other management practices. By contrast, BA has a significant impact on planning, but its effect on measurement is small and not as important. Table 5 shows that both the BI system and BA significantly impact process effectiveness, but the weight for the BI system is significantly higher than for BA.

Control variables
We examined potential heterogeneity of the findings using the control variables identified earlier in this paper by comparing the outer loadings of the measurement model subsets. A comparison of the regression weights was performed using the PLS-MGA approach [21]. The only significant finding was a difference in the R2 for the measurement practice between large (R 2 = 0.224) and small organizations (R 2 = 0.297). This appeared result from the combined effect of a stronger link between the BI system and BA (0.617 versus 0.572), and between analytics and measurement (0.212 versus 0.137) for smaller firms. Overall, the data suggest that smaller firms rely more on analytics to inform their measurement practice.

Discussion
This exploratory research project examined the relationship between the effectiveness of BI system implementation and the effectiveness of CPM practices. We also explored the relative importance of the BI system versus BA.
This study shows a positive and significant relationship between BI system implementation effectiveness and the key corporate management practices of BA, planning and measurement. The strength of the relationship differs among the practices however. BI system effectiveness shows a strong relationship to BA effectiveness (0.581) accounting for 33.6% of its variance. Given that, in this study, the BI system was positioned as an information delivery tool, this finding suggests that BA processes in organizations rely on the BI system, but that other sources of information are also used. Clearly, other factors would contribute to BA effectiveness such as the skills of the analysts and the processes being used. The finding does suggest, however, that the BI system plays a positive role in BA effectiveness.
The total effects calculations show that BI system effectiveness is strongly related to measurement (0.499), planning  (0.566), and process effectiveness (0.444). BA effectiveness, by contrast, shows a weak relationship to measurement (0.204, effect size of 0.042) and to process effectiveness (0.211) but a stronger relationship to planning (0.311, effect size of 0.108). Given that planning is thought to be a management practice characterized by higher levels of ambiguity than measurement, this finding is consistent with information processing theory in that BA practices such as data mining can be used to help reduce the ambiguity present in large amounts of data. Planning and measurement influence process effectiveness to different degrees (0.377 and 0.459, respectively). Measurement has a stronger relationship which might result from the fact that planning occurs less frequently in organizations than does measurement. Yet both function as control mechanisms that help the organization identify the right types of processes to put in place and provide feedback to continually modify processes as needed.
The study reveals that BI system effectiveness is related to the effectiveness of important CPM practices but with different levels of influence. Overall, the BI system shows a stronger influence than the BA function. This might be because BA is relatively new in organizations, or it might be due to the different frequencies with which organizations conduct the different corporate-level management practices. Alternatively, many BI systems now include basic analytic functionalities (i.e., for forecasting and trend analysis), and therefore, it is possible that previously specialized BA functions are being built into new BI system tools.

Implications
The study has several important implications for theory. First, it provides support for the Integrative Model for corporate-level management practices. The study also provides validation of information processing theory in the context of the different uses of information in BI and BA. This is an important issue particularly considering the differences noted between transactional and analytic information delivery [50] and the managerial skill needed to implement systems that support the different information needs. As the use of BA increases in organizations, it would be important to understand the different skill sets, both of IT staff who implement these systems, and of the managers and decision makers who need to interpret the analyses.
The study further suggests that advanced analytics capabilities in modern organizations might call for a revisiting of the concept of media richness. This theory was developed in the late 1980s where the emphasis was on the information carrying capacity of the channel between the sender and the receiver [11]. In modern organizations where BA practices have become more sophisticated, perhaps the notion of media richness can now be expanded to include the interpretation systems of the receiver. That is, sophisticated analytic methods such as data mining or artificial intelligence algorithms might serve the purpose of reducing ambiguity in place of or in concert with media-rich channels of communication.
This research offers several implications for practice, especially for BI stakeholders who are involved in planning, reviewing or implementing BI to support CPM. BI adoption has become widespread as organizations continue to search for ways to support business performance management. Yet, it has been reported that between 70% and 80% of BI projects fail due to inadequate communication between IT and business users about the specific uses of the tools being implemented [19].
For organizations that wish to explore using BI tools to support CPM, this study suggests that a consideration of purpose of the tools-uncertainty versus ambiguity reduction-can help determine the blend between standard BI and more advanced BA packages. An important aspect to consider as well is the level of managerial skill available to implement and support the use of the tools post-implementation.

Limitations and future research
The findings of the study should be interpreted in the light of a few limitations. Because we focused our attention on CPM, we gathered information from senior managers in the participating organizations. Additional research focusing on the middle management layers might shed more insight on the research questions. Moreover, the use of additional variables such as the competence of staff to use BI tools might help to better explore the linkages of BI to process effectiveness. Nevertheless, the study provides an important first step in examining how BI influences CPM practices in organizations and establishes a foundation for future research that might include data collection from managers at different layers in the organizational hierarchy and the use of longitudinal approaches to better understand the specific mechanisms through which BI supports CPM over time.