Towards Handling Uncertainty-at-Source in AI – A Review and Next Steps for Interval Regression

,


I. INTRODUCTION
I NTERVAL-VALUED (IV) data have gained increasing attention across a variety of contexts as they offer a systematic way to capture information complete with an intrinsic representation of range or uncertainty in each individual 'measurement', which is not possible using pointvalues, such as numbers or ranks.Such data may arise due to imprecision and uncertainty in measurement (e.g., sensor data), inherent uncertainty of outcome (e.g., estimations of stock prices [1], climate/weather forecasting [2]), or inherent vagueness or nuance (e.g., in linguistic terms [3]).A growing body of evidence also suggests that subjective judgement of humans (e.g., experts, consumers) may be better represented by intervals [4], [5], where the interval width/size captures the response range as a conjunctive set (e.g., the reals between 2 and 4), or degree of uncertainty with respect to an estimate or rating-a disjunctive set (e.g., confidence interval) [6].
Regression analysis for estimating one or more variables is an important task in data analysis and machine learning under the wider umbrella of AI.While regression is most commonly applied to numeric data, researchers have begun to develop methods of applying (linear) regression analysis to IV data [7].To date, models designed to handle data sets where both independent (regressor) and dependent (regressand) variables are IV have been developed within the frameworks of random sets and symbolic data analysis (SDA) [8], [9].In the random set framework, intervals are viewed as compact convex sets in R, and their interaction is modelled according to set arithmetic [10]- [14].On the other hand, in the SDA setting, the classical regression model is expanded to IV data where intervals are treated as bi-variate vectors [15]- [22].
Current linear regression models for intervals use different reference points, such as the regressors' center values, their lower and upper bounds, or both their center and range (width), to estimate the IV regressand [21].Earlier interval regression approaches [15]- [17] risk 'flipping' the interval bounds, respectively generating a negative interval range, thus breaking mathematical coherence (i.e., violating the definition of interval).The more recent approaches [18]- [21] impose either positivity restrictions on parameters, or design the regression model so as to ensure that the lower bound does not exceed the upper bound.For instance, Neto and Carvalho [18] apply the Lawson and Hanson algorithm (LHA) [23], Wang et al. [19] use Moore's linear combination [24], and Souza et al. [21] the Box-Cox transformation [25] to guarantee mathematical coherence.Although Sun and Ralescu [20] consider positivity restrictions on parameters in their model setting, they do not IRG with respect to range, Ŷ w (≃ 2 Ŷ r ) using the PM method [21].
provide a method to maintain these restrictions.A detailed review and analysis of these models is provided in Section II and the general issue is discussed throughout the paper.An important aspect is that most current approaches, including the above, restrict parameters based on the assumption that an increase in uncertainty 1 of a regressor (i.e., input) leads to greater uncertainty in the dependent variable or regressand.However, there are plausible cases in the real world where this positive correlation does not hold.For example, in cybersecurity, uncertainty around the vulnerability of a system may lead to certainty in the need to invest resources into controlsthus mitigating the risk of the system being compromised, that is, a negative correlation.Similarly, in a multivariate case, consider an economics example; greater uncertainty in future product prices and interest rates may cause more certainty with respect to present consumption choices.
For illustration, Figs.1(a) and 2(a) show a synthetic example of two IV data sets.Here, in Set-1 (Fig. 1(a)), the position (center) and range (uncertainty) of the regressand, Y increases with the position (center) and range (e.g. the uncertainty) of regressor, X-that is, they vary in unison-this reflects the most common case as outlined above, e.g., the price of cars vs their horsepower.In contrast, for Set-2 (Fig. 2(a)), the range (uncertainty) of the regressand, Y increases irrespective of the range (uncertainty) of the regressor, X.This reflects a less common scenario, e.g., the wealth of people with respect to their age.The relationships between the variables in each figure are captured through Interval Regression Graphs (IRGs) in Figs.1(b) and 2(b) respectively.IRGs will be introduced in detail in Section III-B, but for now, note how Fig. 1(b) shows a gentle increase in the range of the dependent variable (2Y r ) for an increase in the range of the independent variable (2X r ) for Set-1, while Fig. 2(b) shows, as expected, no such change for Set-2.The IRGs also highlight the much less drastic increase in range of Set-1's dependent variable that is attributed to an increase in the position of the independent variable (X c ), compared to Set-2, in which variance in regressand range (2Y r ) is attributed solely to regressor position (X c ).
Aspects such as the above highlight the complexity but also the potential of IV data in directly capturing uncertainty and range at-source, making it accessible for reasoning.With intervals encoding additional information by comparison to numbers, the choice of a 'universally best' analysis approach, as can often be found for numeric cases, is however highly challenging.Instead, the selection of appropriate techniques for a given IV data set-based on its properties-is essential.Ferson et al. in [26], discuss the complexity of analysing realworld IV data in detail, elucidating the value of focusing on key properties for data sets to inform the choice of methods, such as whether the intervals exhibit strong overlap (i.e., 'puffy' [26]: wider, more uncertain intervals) or very little overlap (i.e., 'skinny' [26]: narrow, less uncertain); whether there are outliers; whether intervals are highly scattered, densely placed or even nested within each other.Beyond these properties, more fundamental questions arise from the semantic meaning of the given intervals, as already alluded to above; whether they represent an epistemic, disjunctive set, with the interval describing the range within which a correct answer is contained (this is arguably the most common use of intervals in science, cf.confidence intervals), or indeed an ontic, conjunctive set with a true range, such as the real numbers between 1 and 5, or a time span [5], [27].For conciseness, we do not discuss the latter further in this manuscript, but will revisit it, and its impact on the analysis of AI and machine learning more generally, in future publications.
At this stage, we also note that of course one can go beyond intervals to model uncertainty, leveraging distributions with a probabilistic or fuzzy interpretation, depending on the dis-or con-junctive nature of the information.Focusing on intervals a priori provides advantages, such as increasingly established pathways to collecting data (e.g., [5]), while also laying foundations for more advanced methods.For example, in fuzzy set theory, α-cut decomposition provides a direct translation mechanism between intervals and fuzzy sets.
This paper makes four principal contributions: 1) Review of the state-of-the-art: In view of the significance and use of intervals as a fundamental data-type in AI, the paper presents a detailed review of all well-known stateof-the-art linear regression approaches based on vector representation of data sets where both independent and dependent variables are IV.2) Extensions of current methods: As part of the review of the methods, this paper puts forward extensions to the Linear Model (LM) [20], enhancing its robustness efficiently for real-world applications.Specifically, we propose a dynamically selective approach to applying restrictions, letting the model's parameters vary freely unless the mathematical coherence is being compromised.To support the broader adoption of intervals as a fundamental data-type in AI, extant regression approaches and their performance are analysed using synthetic and real world data sets capturing various common and features of intervals, such as 'puffy' and 'skinny' intervals [26].The resulting insights are distilled, for the first time, to guide readers in selecting the regression algorithms appropriate to their data.Further, we provide open-source software at http://lucidresearch.org/software, offering, for the first time, direct access to IV regression approaches, with a view to facilitating adoption and replication.4) Explainability and visualization: The explainability of numeric linear regression models is one of their primary assets as an AI technique.This paper introduces a novel visualization approach for IV regression-the Interval Regression Graph (IRG)-which for the first time provides the means to visualize and succinctly communicate the relationship in terms of both position and range between IV regressor and regressand. 2he paper is organized as follows: Section II reviews linear regression methods based on vector/matrix representation of intervals in the regression process.Section III introduces the interval regression graph (IRG) and discusses adaptations of the LM method [20] for real-world applications, followed by IV data sets (synthetic and real-world) and evaluation metrics.Section IV demonstrates the estimation performance of all regression models with respect to both synthetic and realworld data sets.Section V discusses suitability of the existing regression approaches to salient features of IV data, distilling insights into practical guidance for algorithm-selection.Lastly, Section VI concludes the paper with future works.Table I presents a list of acronyms and notation used in this paper to assist the reader.

II. BACKGROUND AND STATE OF THE ART
A closed interval a is characterized by its two endpoints, a − and a + with a − ≤ a + where a − and a + are respectively the lower and upper bounds of a [29].Generally, a is denoted as a = [a − , a + ], and its range is given by a w = |a + − a − |.Alternatively, the same interval can be represented as a = [a c − a r , a c +a r ] with its center, a c = (a + +a − ) 2 and radius, a r = a w 2 .In this section, we review linear regression methods for full IV data sets (i.e.where both dependent and independent variables are intervals) where intervals are treated as bi-variate vectors in the regression process.We note Y = {y 1 , y 2 , . . ., y n } as an IV regressand with n intervals, where where each X j has also n intervals, Thus, there are p pairs of (Y, X) in the data set.In addition, the estimated value of Y is given by Ŷ = { ŷ1 , ŷ2 , . . ., ŷn }.
Since about the year 2000, a variety of linear regression approaches for full IV data sets have been put forward.We briefly review them below in chronological order.

A. The Center Method (2000)
Billard and Diday [15] proposed a linear regression model, known as the Center Method (CM) in 2000, to fit the center of both regressand and regressors, and then apply this model to the lower and upper bounds of regressors to separately estimate the upper and lower bounds of the regressand.Equation (1) defines the CM regression model, where y − i and y + i are the lower and upper bounds of y i .Similarly, x − ji and x + ji are the the lower and upper bounds of x ji .β c j ∈ R are the regression coefficients for the center estimation, 0 ≤ j ≤ p, p is the total number of regressors while ϵ − i and ϵ + i are error terms.Both lower and upper bound models of (1) use the same coefficients, therefore, they can be captured together using (2).
where y c i and x c ji are the center of y i and x ji respectively and ϵ c i = (ϵ − i + ϵ + i )/2 is error term.Equation (2) can also be expressed for the total set of n observations in the matrix format as Y c = X c β c + ϵ c , where The coefficients, β c are estimated using the least squares (LS) method [30], defined in (3).
The CM model is simple, as it follows a standard regression approach applied to the center of the intervals and then the resulting model is applied both to upper and lower endpoints.However, by doing so, it risks and often fails to maintain mathematical coherence [17], that is, the estimated lower bound of y i becomes larger than the estimated upper bound.

B. The MinMax Method (2002)
Billard and Diday [16] presented the MinMax method in 2002 which directly uses the lower and upper bounds of the regressors to separately estimate the lower and upper bounds of the regressand.Equation ( 5) defines the MinMax regression model for the lower and upper bounds of the regressand, where β − j and β + j are regression coefficients for the lower and upper bounds respectively (β − j , β + j ∈ R, 0 ≤ j ≤ p).In matrix notation, the lower bound model of (5) with all n observations is expressed as Like the CM method, (6) estimates the coefficient (β − ) for the lower bound model of (5) using the LS method [30].
Now, using the estimated β− , (7) estimates the lower bounds for the regressand, Y .
Similarly, the upper bound model of ( 5) is expressed as Y + = X + β + + ϵ + , where Equation (8) estimates the coefficients for the upper bound of (5) by the LS method [30].
In the MinMax method, the use of two separate models to estimate the lower and upper bounds of the regressand improves the model fitness and interpretation compared to the CM method [17].However, it still does not guarantee mathematical coherence on the estimated bounds [17].Further, its estimation performance can be reduced if there is no dependency between the bounds of regressand and regressors [21].

C. The Center and Range Method (2008)
Neto and Carvalho [17] extended the CM method [15] in 2008 to the Center and Range Method (CRM), by considering both interval center and half the 'range' (i.e., the radius) of the regressor and regressand variables.They build two separate regression models-one for the centers and other for the ranges (or radii).Equation ( 10) is used to model the interval centers.
where y c i and x c ji are the centers y i and x ji respectively and ϵ c i is error term.The CRM method follows the same matrix format of the CM method to represent the center model of (10) and applies the same strategy of the CM method to estimate the coefficients β c j (see (3)).It then estimates the center values for the regressand, Y using the estimated βc j by (11).
Similarly, the interval ranges are modelled by the CRM method using (12), where y r i and x r ji are the radius of y i and x ji respectively.β r j ∈ R are the coefficients for the range estimations, 0 ≤ j ≤ p and ϵ r i is error term.Again, following the same matrix notation, the coefficients, β r j are estimated by (13), and using the estimated βr , (14) computes the ranges of the regressand, Y .
Finally, the CRM method estimates the bounds of the regressand from the estimated ŷc i and ŷr i using (15).
The CRM model provides better estimation when there is a linear dependency between the ranges of regressand and regressors [18], [21].If this is not the case, it does not ensure mathematical coherence as ( 14) can produce negative y r i [21].
D. The Constrained Center and Range Method (2010) Neto and Carvalho [18] later refined the CRM model [17] (see Section II-C) in 2010 to ensure mathematical coherence.The resulting approach is known as the Constrained Center and Range Method (CCRM), where a positivity restriction is enforced on the coefficients which are estimated with respect to the relationship of the radii of regressand and regressor variables.In other words, the overall estimation process in the CCRM method remains the same as for the CRM model (see (10) to (15)) with positivity constraints on the range coefficients β r j .The CCRM model uses an iterative algorithm proposed by Lawson and Hanson in [23] to ensure β r j ≥ 0. For more detail, we refer the reader to [18].Finally, the CCRM method estimates the bounds of the regressand using (15).
Neto and Carvalho recommend to apply the CRM model in all cases, only adopting the CCRM method as a suitable strategy when the CRM method fails to maintain mathematical coherence, as the use of the CCRM model can lead to biased estimation outcomes [18].In particular, the positivity restriction within the CCRM method forces any negative range coefficient to 0 (at the same time updating the remaining range coefficients), in turn leading to potentially biased estimation outcomes and poor model fitness [18].

E. The Complete Information Method (2012)
Wang et al. [19] presented the Complete Information Method (CIM) in 2012 with a focus on considering all the information contained within the intervals.The CIM model considers each interval observation of regressand and regressor variables as a hyper-cube (if a single-valued regressor and associated regressand are considered, each observation effectively reflects a rectangular, as visualized in Fig. 4), and the regression model is built on these hyper-cubes.Further, it adopts Moore's linear combination algorithm [24] to ensure the mathematical consistency of the bounds.In this model, Y is presented by a linear combination of X j , where β j represents the coefficients with 0 ≤ j ≤ p and 1 n is an n-dimensional column constant vector of ones.Equation ( 17) defines the lower and upper bound estimations of Y . where, Using (16), the LS method [30] of (17) generates the following linear system, where, Here, ⟨X a , X b ⟩ is the inner product between X a and X b (0 ≤ a, b ≤ p and X 0 = 1) which is computed using (19).18), the coefficients β can be estimated by the inverse of matrix M as defined in (20).
Thus, using β and τj , the lower and upper bounds of the regressand, Y can be estimated by (17).
The CIM method satisfies mathematical coherence of the estimated bounds of the regressand using τ j where its value (1 or 0) depends on the sign (+ or -) of βj [19].

F. The Linear Model (2015)
Sun and Ralescu [20] developed the Linear Model (LM) in 2015 based on the affine operator in the cone C = {(x, y) ∈ R 2 |x ≤ y}, aiming to maximize model flexibility while preserving its interpretability.This model considers both lower and upper bounds of regressors and their ranges for estimating the bounds of the regressand.In this model, y i is expressed as a linear transformation of x ji such that, where the regressor coefficients, α j , β j , η ∈ R and γ j , θ ≥ 0, j = 1, 2, ..., p. Equation ( 21) can also be written in the following matrix form as Y = Xβ + ϵ, where Again, the LS estimate of the coefficients, β are computed by (22) and ( 23) estimates the bounds of the regressand, Y .
Even though the authors assume positive constraints on range coefficients (i.e., γ j ≥ 0 and θ ≥ 0) in ( 21) to maintain mathematical coherence, the actual model does not ensure compliance with these constraints.As a result, (22) can result in negative range coefficients (γ j , θ < 0), which may lead to flipped interval bounds (ŷ − i > ŷ+ i ) through (23).The authors do not discuss how to maintain these constraints in practice, though they expect that if any of the estimates of γj and θ turns out to be negative, forcing it to be positive may lead to poor fitness of the LM model [20].

G. The Parametrized Model (2017)
Souza et al. [21] proposed the Parametrized Model (PM) in 2017, which also builds two different models for the regressand bounds.This approach extracts the best reference points from the regressors and uses them to build linear regression models for both lower and upper bounds of the regressand, unlike earlier approaches that use specific reference points on the regressors, such as center, range, interval bounds.
In this method, an interval is considered as a line segment, useful to reach all points within it.For instance, given an interval a, any point q ∈ a can be computed as q = a − (1 − λ) + a + λ, 0 ≤ λ ≤ 1.By setting λ, a is turned into a single point.Hence, when λ = 0, q = a − (lower bound of a) and when λ = 1, q = a + (upper bound of a).Similarly, q = a c (center of a) when λ = 0.5.Utilizing this concept, the PM method specifies the linear regression models for the lower and upper bounds of Y in (24).
In matrix notation, the lower bound model can be expressed for all n observations as Y − = X * β − + ϵ − , where .
The LS estimate of the coefficients for the lower bound model, β − is computed by (26).
The matrix expression follows the same pattern for the upper bound model, Y + = X * β + + ϵ + , and the LS estimate of the coefficients for the upper bound model, β + in defined in (27).
Finally, using β− and β+ , the lower and upper bounds of Y are estimated using (28).
The PM method does not automatically guarantee the mathematical coherence of the bounds [21].To avoid flipping the interval bounds, the approach estimates the range of Y using (29) before performing the regression.
If all estimated ranges are positive (ŷ w ∈ Ŷ w ), the model automatically ensures mathematical coherence.However, if at least one of the estimated ranges is negative, it applies the Box-Cox transformation [25], extended to intervals by the authors [21] (defined in ( 30)), to transform the regressand so that the desirable coherence is achieved by the PM method.
where y k i is the transformed interval of y i .k 1 is any real value and k 2 maintains the following restriction:

III. DATA AND METHODOLOGY
In Section II, we discussed linear regression models for intervals considering their vector/matrix representation.Table II provides a summary of these models and their features.In this section, we initially put forward an extension to the LM method [20] which ensures mathematical coherence, thus making it more suitable for real-world data.Next, we introduce  [17] Center and radius (half of range) No No CCRM [18] Center and radius (half of range) No Yes with positivity restrictions on range coefficients CIM [19] All points within intervals Yes Yes with Moore's linear combination algorithm [24] LM [20] Lower and upper bounds, and range No Yes with positivity restrictions as proposed in this paper PM [21] Any point within intervals Yes Yes with the interval Box-Cox transformation [25] a novel approach to intuitively visualizing IV regressionthe Interval Regression Graph (IRG).IRGs are designed to facilitate the interpretation and communication of the regression model and thus the relationship between an independent and dependent variable of interest.The latter is a crucial asset of traditional (numeric) regression models, accounting for a substantial part of their utility and popularity.Going beyond the traditional visualization of numeric regression, IRGs communicate the relationships of both position and range (uncertainty) of IV regressor and regressand.Following the introduction of IRGs, we introduce both synthetic and real-world data sets to support the empirical exploration and analysis of the regression approaches in Section II.Finally, we discuss evaluation metrics employed for performance assessment during the remainder of the paper.

A. Extension of the Linear Model [20]
As pointed out in Section II, the LM method [20] considers positivity restrictions on the range coefficients to consistently estimate the regressand bounds.However, it neither guarantees these restrictions through its model setting, nor provides algorithmic solutions to maintaining them.Thus, to make the model suitable for practical real-world deployment, we put forward two alternative extensions to the LM method: (1) universally forcing positive restrictions on parameters as [20] suggested-that is proposing a constrained LM method, and (2) enforcing restrictions only when needed to avoid unnecessary estimation bias [20]-that is, proposing a weakly constrained LM method.
1: Apply the LM method at (22) on Y and X j (1 ≤ j ≤ p) to estimate β * and βw 2: if θ < 0 or γj < 0 where θ, γj ∈ βw , then 3: Apply LHA [23] to make θ and γj positive 4: end if 5: Estimate Ŷ − and Ŷ + using (23) (1) We propose a constrained LM method (LM c ) following the notion of the CCRM method [18] where the Lawson and Hanson algorithm (LHA) [23] is applied to universally impose the positivity restriction on the range coefficients, γ j and θ.Algorithm 1 presents this constrained LM variant, LM c .Note that universally (or blindly) applying the restrictions for maintaining mathematical coherence risks poor performance in terms of model fit [20].
(2) To minimize the impact on performance, we also develop a more weakly constrained LM method (LM w ), in line with the rationale of the PM approach [21] to avoid unnecessary bias in the estimation results due to universally restricting parameters.Here, we first regress the range of the regressand variable (y w i ) on the range of regressors (x w ji ) before performing the estimation using (29) as the estimated range values of the regressand guarantee mathematical coherence of the estimated bounds if they are positive [21].Therefore, if all estimated range values are positive (ŷ w i ≥ 0, ∀i), we compute the least squares estimation of ( 22) without any parameter restrictions.In contrast, if any of the estimated range values are negative (ŷ w i < 0, ∃i), we enforce positivity restrictions on γj and θ using the Lawson and Hanson algorithm (LHA) [18], [23].Algorithm 2 summarizes this adapted approach, LM w .

B. The Interval Regression Graph (IRG)
Visualization of traditional regression models provides a powerful mechanism for interpretability and communication of the relationship between numeric variables.While intervals provide a richer model, enabling the capture of uncertain information, the crucial interpretation and communication of any insights is complex, yet just as vital as for the numeric case.We introduce a novel 3D visualization approach, the interval regression graph (IRG) which succinctly and clearly captures the change in a regressand's key features (center and

Algorithm 3 Interval Regression Graph (IRG) Generation
Input: An IV regression model (IVRM).The IV regressor's (e.g., from the original data set) minimum range and maximum range are rangeX min and rangeX max , as well as its minimum center and maximum center coordinates are centerX min and centerX max .Output: Two IRG plots mapping the regressor to the regressand's center, Ŷ c and range, Ŷ w .1: Generate the set X w of p discretizations of the interval [rangeX min , rangeX max ] a 2: Generate the set X c of q discretizations of the interval for each discretized X c j , 1 ≤ j ≤ q do 5: Compute end for 11: end for 12: Generate Ŷ c IRG plot with Ŷ c on the vertical axis, X w on the bottom left axis and X c on the bottom right axis 13: Generate Ŷ w IRG plot with Ŷ w on the vertical axis, X w on the bottom left axis and X c on the bottom right axis 14: return IRG plots for Ŷ c and Ŷ w a Note that the IRG generation algorithm is designed to allow for the visualization of both linear and non-linear relationships between variables.In this paper, we focus on linear relationships/regression models only.range) for given changes in a regressor's key features (center and range).The IRG visualizes the correlation between a regressor's and regressand's position (or range), based on a given regression model [31].Note that in this paper, for conciseness, we only introduce and focus on IRGs generated for visualizing the relationship between one regressor and one regressand.Algorithm 3 presents the pseudocode for constructing IRGs for a regressand's center and range with respect to a regressor's center and range-for a given regression model.
To illustrate the IRG and its utility, consider the IV Set-1 (Fig. 1(a)), presented in the introductory section.Figures 3(a) and (b) individually show the two different aspects of the IRG for Set-1 based on the PM regression method.The bottom-left and bottom-right axes show the regressor's center (X c ) and range (X w ≃ 2 Xr ) respectively.In Fig. 3(a), the vertical axis denotes the regressand's estimated center ( Ŷ c (solid plane)), while in Fig. 3(b), it reflects the regressand's estimated range ( Ŷ w ≃ 2 Ŷ r (dotted plane)).
Interpreting these figures, we can see how Fig. 3(a) visualizes that the regressand's center, Ŷ c increases with respect to increasing values of both the regressor's range, 2X r (or X w ) and its center, X c .Fig. 3(b) shows how the regressand's range, 2 Ŷ r also increases with respect to both increasing values of the regressor's range, 2X r and center, X c , and that it does so at a greater rate in each case than does Ŷ c .For

C. Data Sets
We use both synthetic and real-world IV data sets to investigate the performance of linear regression methods.The synthetic data sets are used to explore and visualize the sensitivity of the regression models with respect to different properties of the interval data, whereas the real-world data sets are used to investigate their suitability to real-world scenarios.All data sets are described below.
1) Synthetic Data Sets: We specifically design a series of synthetic IV data sets to investigate the behavior of the regression models.First, Set-1 and Set-2, shown at the beginning of the paper (Figs.1(a) and 2(a)), are used to explore the relationship between regressor and regressand with respect to the primary features (i.e., center and range).Second, an additional nine sets of intervals (Set-3 to Set-11), exhibiting different properties are used: we use three sets of skinny intervals (Set-3 to Set-5) with smaller range 3 and three puffy sets with wider range (Set-6 to Set-8) having features of disjointedness, some, and full overlap.We also consider three mixed cases (Set-9 to Set-11), containing puffy and skinny intervals in separated, partially overlapping and nested states.All of these data sets are produced within the range of 0 to 10.They are given in Table VII (a) to (k) in the Appendix including the mean (µ) and standard deviation (σ) of the center and range of intervals.Figure 4 graphically presents the data sets, Set-3 to Set-11.
2) Real-world Data Sets: We focus on three real-world IV data sets with different properties and arising in different application contexts, i.e., medical, consumer food ratings/marketing, and cyber-security, reflecting the broad applicability of IV data in real-world applications.
The first data set represents medical observations and includes interval-valued systolic and diastolic blood pressure (BP) data of 59 patients of the Hospital Valle del Nalón in Asturias, Spain [11], [14].
The second data set presents IV consumer ratings of eight different (UK market) snack-food products [32].In this case, 40 consumers rated each product-using 'DECSYS' interval survey software [33]-on six different attributes: visual appeal, value for money, healthiness, taste, branding, and ethics, along with their overall purchase intention (i.e., how likely they were to buy each product).
The third and final data set captures assessments by cybersecurity experts of the vulnerability of system components in a real-world scenario.Specifically, 38 cyber-security experts at the UK CESG (Communications-Electronics Security Group) assessed a range of components (referred to as hops) that are commonly encountered during a cyber-attack [34], [35].They rated the hops on overall difficulty for an attacker to either attack or evade them, as well as rating each on several attributes (seven for attack, and three for evade) that might affect this difficulty.The experts provided their IV ratings on a scale from 0 to 100.For more details of these data sets, see [11], [14], [32], [34], [35].

D. Evaluation Metrics
Evaluation poses one of the key challenges for IV data analysis and AI more broadly.While 'quality of fit' for numeric data is straightforward, the same is not the case for intervals.Comparing the latter requires application-led assumptions (on the nature of the intervals and whether for example interval size is more important than position), which in turn feed into similarity measures or arithmetic means of comparison.We do not delve further into this challenge in this paper, instead following three fairly standard metrics to evaluate the performance of the linear regression models.Each of the evaluation metrics is defined with respect to the regressand (samples), Y = {y 1 , y 2 , ..., y n } and its estimated value(s), Ŷ = { ŷ1 , ŷ2 , ..., ŷn }. 1) Root Mean Squared Error (RMSE) estimates error by considering the average deviation of the estimated values from the actual ones [18].The RMSE for the lower and upper bounds of Y is defined as, where a smaller RMSE value indicates a better fitted model.
2) Mean Absolute Error (MAE) calculates the average of absolute difference between the actual and estimated values [36].
For the upper and lower bounds of Y , the MAE is defined as, where 0 ≤ M AE − , M AE + ≤ R + .Again, a lower value of MAE implies a better fitted model.
3) Mean Magnitude of Relative Error (MMRE) measures the mean of dispersion of the actual and estimated values with respect to the actual values [37].It is calculated as follows: where 0 ≤ M M RE ≤ R + .In this case, the lower the MMRE, the better the model.

IV. RESULTS
We explore the performance of existing regression approaches together with two proposed variants of the LM method (LM c and LM w ) for synthetic and real-world IV data sets (see Section III-C).Their performance is assessed using different metrics described in Section III-D.All these regression methods are implemented in Matlab and the full source code will be available at http://lucidresearch.org/software with the publication of the paper.

A. Performance Analysis with Synthetic Data Sets
In Section III-B, we briefly introduced the IRGs using IV Set-1 (Fig. 1(a)).We expand on this example here, reporting performance of all regression models covered above with this set, as well as for Set-2 (Fig. 2(a)).
As discussed earlier, Set-1 shows the more commonly considered and intuitive scenario where uncertainty and position in regressor and regressand vary in unison.On the other hand, Set-2 presents a less commonly considered scenario, where the uncertainty of the regressand increases irrespective of the uncertainty of the regressor.Said differently, for Set-2, uncertainty and position of the regressand vary quasi exclusively with respect to the position of the regressor variable.
Tables III and IV present the fitness results for all regression approaches for Sets 1 and 2, respectively.These show that the PM method provides the best fitness in each case, though the MinMax method provides almost equal fitness for Set-2 (they do differ with respect to M AE + ).For conciseness, we focus further discussion of the regression results on two of the regression approaches: the CRM-representing an earlier and more basic approach which does not guarantee mathematical coherence, and the PM-the most recent approach, which does.IRGs for all methods are shown in the appendix (Figs.18 and 19).We independently present the relationship of the center and range of the test regressor on the center (subfigures (a)) and the range (subfigures (b)) of the regressand, for the CRM and PM methods in Figs. 3,       the center of the regressor, and the estimated range of Set-1 is influenced only by the range of the regressor.Thus, the CRM method is not in line with our expectation for the Set-1 data set.The rest of the methods also do not perform according to the expectation (see Appendix, Fig. 18).Further, the IRGs in Figs.6(a) and (b) show how, according to the PM model, both center and range of the regressand of Set-2 vary solely with respect to the center of the regressorin line with expectation.The IRG in Fig. 7(a) visualizes how, using the CRM method, the estimated center of Set-2's regressand is similarly positively influenced by the changes in the center, but not the range of the regressor-as expected.However, in contrast to the PM model outputs, the CRMderived IRG for the same data, shown in Fig. 7(b), indicates no strong relationship between regressand range and either facet of the regressor.If any relationship is present at all, then this is a negative effect of the range of the regressor on that of the regressand-these results do not fit those expected when viewing the data.Fig. 19 (in the Appendix) visualizes that the MinMax method shows similar performance to the PM method.The remaining methods perform less well.
Next, we will briefly discuss equivalent analysis and model visualizations, using the IRGs, for an additional nine synthetic data sets (Set-3-Set-11), each designed to reflect a key property regularly encountered in IV data sets.This will highlight how different data sets-with different features-are differently suitable for different regression methods, as later results will expand upon in more detail.For data Set-3 to Set-11, all shown in Fig. 4, we briefly summarize individual performance results across methods below before discussing performance more broadly.Table V(a) -(i) present the fitness of regression methods for each of these data sets.
• Set-3 (All disjoint skinny intervals): The PM and LM w methods produce the best fit for all disjoint skinny intervals.
The LM c method results in equally good fit well with respect to two metrics (RM SE − and M M RE).• Set-4 (Some overlapping skinny intervals): Again, the PM and LM w methods produce the best fit.The LM c method also gives the same fitness with respect to three metrics (RM SE − , M AE − , and M M RE).The fitness of other methods improves with some overlapping of intervals.• Set-5 (All overlapping skinny intervals): The PM and Min-Max methods come out as the best approach.Among others, CRM, CCRM, LM c and LM w methods produce quite similar results with respect to some of the evaluation metrics.
• Set-6 (All disjoint puffy intervals): The PM method gives the best fit, though the LM c and LM w methods also produce quite similar results for puffy intervals.Overall, the PM, and LM w (and often LM c ) methods perform best and result in reliably good-and almost identicalfitness for disjoint skinny or puffy intervals.With growing overlap among intervals (skinny or puffy), model fitness by the CRM, CCRM and MinMax methods generally improves.Nevertheless, they consistently fail to reach the performance levels of the PM and LM methods.For the mixed sets with puffy and skinny intervals, where these are disjoint or exhibit some overlap, both PM and both variants of the LM method result in similarly good (best) fit.Others, particularly the CRM and CCRM approaches, give nearly identical results for some metrics, for all overlapping, respectively nested, mixed cases.In all cases, the estimates by the CIM and CM methods deviate substantially from the actual values and hence they come out as poor performers.
As well as the empirical analyses reported above, we (b) Model Performance Fig. 8: Graphical presentation of systolic and diastolic BP data set [11] and performance of different regression models as to evaluation metrics using this set.carried out additional experiments going beyond the scope of this paper-exploring the impact of differences in mean interval size/range, differences in the standard deviation of the mean size/range, and the variation in interval centers (i.e., dispersion), on model fitness or estimation errors [38].Results suggest that estimation errors tend to grow for all regression methods with respect to both higher standard deviation of ranges and more distant placement of intervals (centers) [38].
We will review this aspect with real-world data in future.

B. Real-world Data Sets
In this section, three real-world IV data sets from the application contexts described in Section III-C, are used to explore and evaluate the behaviour of different regression methods with respect to the mixed set of features of the data (in the sense of the previous section) present in these sets.The fitness of each method is compared using the same evaluation metrics (i.e., RMSE, MAE, MMRE) as before.
In addition, the larger degrees of freedom-compared to the synthetic data-of these real world data sets, permit us to empirically determine a measure of the variability associated with each performance value, utilizing a non-parametric resampling approach [39].We therefore report bootstrapped (percentile) 95% confidence intervals [39] [40] associated with each error measure, for each method and all three real-world cases.Note here that although many of the confidence intervals overlap, this does not necessarily imply non-significance of differences between the individual regression approaches' error measures in question.As these were calculated on the same resampled data sets, the measures on each instance were non-independent, and so it is possible for variability evident in the CIs to represent common inter-simulation variance between measures, while for example one was a reliably better performer than another.
1) Systolic and Diastolic BP Data Set: For this data set, we regress the IV diastolic BP (regressand) on those of the systolic BP (regressor) to estimate the bounds of diastolic BP. Figure 8 presents this data set and the associated performance of the regression methods.
Figure 8(b) shows that the PM, LM w , and LM c methods produce comparable performance in estimating the regressand bounds.With respect to some of the evaluation metrics (RMSE + and RMSE − , and MAE + ), the PM produces the best estimation, whereas the LM w and LM c methods perform better for MAE − ; all these three methods perform equally best for MMRE.Fit for the CRM, CCRM and MinMax methods for these data is also better according to MMRE.In this particular case, the CM and CIM methods result in the worst fit.
Along with evaluating model fitness, we also explore the relationship between the uncertainty (and position) of the systolic BP and that of the diastolic BP in Fig. 9.Note that for space saving, we only present here the IRGs using the PM method.Figures 9(a) and (b) show that the center of systolic BP is positively correlated with the center of the diastolic BP and the range of systolic BP exhibits the same trend, i.e., increasing rapidly with the increasing range of the diastolic BP.These figures also highlight that the center of systolic BP decreases, at a lesser rate, as the range in diastolic BP increases, and vice versa, i.e., the range of systolic BP decreases as the center of diastolic BP increases.
2) Food Snacks Purchase Intention Data Set: With this data set, we first remove outliers and then separately regress the overall purchase intention (regressand) on each of the six attributes of snack-bars (i.e., visual appeal, value for money, healthiness, taste, branding, and ethics).Due to page limitations, we present regression results only with respect to two attributes-ethics and taste (see Figs. 10 and 12).The estimation results in Figs.10(b) and 12(b) show that the PM, LM w , LM c , and MinMax methods produce the best fit with respect to one or more indices.However, bootstrap confidence intervals indicate substantially greater proximity and variability in these estimates than for the previous BP data set.This is especially true for the PM method for the data set representing taste and overall purchase intentiondemonstrating how in cases where transformation is required to maintain mathematical coherence, this can have a large  (b) Model Performance Fig. 10: Graphical presentation of ethics and overall purchase intention [32] and performance of different regression models as to evaluation metrics using this set.detrimental impact on model fit, as discussed earlier.
Beyond the evaluation of model fit, we also use the IRGs to explore how the selected attributes of snack-bars influence overall purchase intention, as discussed in [32].We construct IRGs to show the impact of two attributes-ethics and taste on those of the overall purchase intention.Again, to save space, we only show the IRGs corresponding to the PM method.Figure 11 shows how the position and uncertainty of ethics affect those of overall purchase intention.The center of overall purchase intention, Ŷ c increases with the center of ethics, X c , as well as its range, 2 Xr .Similarly, uncertainty around overall purchase intention, 2 Ŷ r is positively influenced to some extent by both position, X c and associated uncertainty of the ethics, 2X r , though much more strongly by the latter.The IRGs (shown in Fig. 13) relating to taste also suggest implications for both position and width on each aspect of overall purchase intention.However, comparing the IRGs for taste and ethics, we observe that the position and uncertainty of two attributes tend to have distinctive impacts on overall purchase intention.For example, increasing uncertainty in ethics (together with increasing center values) increases position of overall purchase intention, whereas this tends to decline for rising uncertainty in taste (while also increasing for higher center values).
These results, which are uniquely captured by the IV data, are made visible via the IRGs.We emphasise that the regression analysis using only two variables as shown here, while a useful illustration, is limited and a broader analysis would be expected to extract real-world insights.We point the interested reader to [32], where a more detailed analysis-but based on a discrete representation of the data, is conducted.
3) Cyber-security Data Set: Section III-C provided an overview of this data set.Here, we only consider the evade How complex is the job of providing this kind of defence?Availability of How likely is that there will be publicly available information information that could help with evading defence?Maturity How mature is this type of technology?Overall rating Overall, how difficult would it be for an attacker to do this?hops as with three attributes these provide a concise basis for presenting the results within the constraints of this paper.For each evade hop, 38 experts gave ratings on following three attributes: complexity, availability of information and maturity.
They were also asked to provide an overall rating on how difficult it would be overall for an attacker to evade a given hop.These ratings were given on a scale from 0 to 100.Table VI summarises the attributes for the evade hops.We explore how well the regression methods estimate the IV overall difficulty ratings for selected evade components, based upon the three interval-valued attribute ratings, and by comparison with experts' actual overall ratings.For conciseness, we present two evade hops -number 4 and number 22, shown in Figs. 14 and 15 respectively.Figure 14 (b) shows that the PM method appears to be the best performer for Evade-4 with respect to four evaluation metrics (RMSE − , RMSE + , MAE + , and MMRE) whereas the LM w method produces the joint best fit on MAE − .The LM c method also performs very close to the LM w method.Again, the CM method provides poor fit to the data.As for the Evade-4 data set, Fig. 15 (b) displays similar results for Evade-22.Particularly, Fig. 15 reveals the PM method as the best approach in terms of most metrics.Further, the LM w , LM c , CRM, and CCRM methods come out as suitable measures as to either RM SE + or M AE + .We provide all estimated coefficients for both synthetic and real-world data sets in Appendix (see Table VIII and IX).Note here again that bootstrap confidence intervals indicate substantially higher variability in model fit than for the first data set.However, in contrast to the second data set-for which the PM method often showed substantially higher upper confidence bounds than the other methods-the PM method's upper confidence bounds are now often the lowest, or nearly lowest.This may indicate relatively few cases in which transformation was required to maintain mathematical coherence for the PM method with this data set, avoiding the associated model fitness penalties.
Across the experiments, confidence bounds for the PM and the LM methods are overall fairly similar.For the BP data,  (b) Model Performance Fig. 12: Graphical presentation of taste and overall purchase intention [32] and performance of different regression models as to evaluation metrics using this set.The only exception where the PM method shows wider variability in CI bounds than other approaches is for the taste vs overall purchase intention data set, Fig. 12.This higher variability in CI bounds may be due to greater incidence of very wide and scattered intervals within the data set, resulting in increased application of the box-cox transformation and poorer fit across a larger subset of samples/simulations. Figure 20 in Appendix shows the total number of times across the 10000 simulations the regression approaches (CCRM, PM, LM c , and LM w ) apply the positivity restrictions/regressand bound transformations to maintain coherence for the real-world data sets.Figure 20 reveals that for the taste vs overall purchase intention data set, for a significant proportion (22.85% of total 10000 simulations), positivity restrictions/transformation of bounds are applied by the PM and LM w methods, compared to other sets.Exactly why the PM method is more strongly impacted than the LM w method for this case requires further research which we will address in a future publication.
V. DISCUSSION ON SUITABILITY OF EXISTING REGRESSION APPROACHES This section discusses the suitability of different regression techniques with respect to interval data sets with different properties (e.g., skinny, puffy or mixed), and based upon the results of the analyses conducted above.The experiments consistently show that the PM and LM w (and often LM c ) generally produce the best model fit-with reliably good performance for skinny/puffy or mixed sets of intervals, when these are disjoint or overlapping.With increasing overlap among intervals, estimation errors also decrease for the CRM, CCRM and MinMax methods, but not generally so far as to equal the performance of either PM or LM methods.In all cases, both MinMax and CRM require checking as to whether they maintain consistency of interval bounds.The real-world data experiments largely substantiate the synthetic experiments in terms of relative model performance.
Results reveal that the LM w method produces better fitness than its permanently constrained sibling (LM c ) in most cases, as the weaker constraining approach lets model parameters vary freely unless mathematical coherence is being compromised.Both approaches generate the same outcome when the coherence of the bounds does break down and LM w effectively reverts to LM c .Similarly, the PM approach may produce much poorer fit in cases where it requires the Box-Cox transformation [21], [25] of the regressand to restore coherence of the bounds.We provide such an example in Appendix (see Fig. 17), and we also hypothesise that this is the cause of the high upper 95% bootstrap confidence bounds in relation to this model's performance on our second real-world data set.Generally, both CM and CIM methods yield poor model-fit.
Having said that, the PM and LM models (particularly its weakly-constrained form, LM w ) were overall the most widely suitable regression approaches.Nevertheless, as we discuss throughout, better performance can be found for data sets following specific properties.To help researchers choose the most suitable method for their data, Fig. 16 provides a flowchart to summarise the findings presented.The figure provides a recommendation on suitability of regression approaches as to different features of the intervals in given data sets.
In addition to comparing model fitness among regression approaches, we have also visualized the relationship between IV variables with respect to their key features (i.e., center and range) by introducing the IRG.The IRG provides a powerful visual tool, akin to the 'regression line' of traditional numeric regression, offering rapid, intuitive insight into the data.While the IRG can be generated for all regression models, irrespective of the regression method, we generally favour the PM method as consistently producing strong model fit.(b) Model Performance Fig. 14: Graphical presentation of Evade-4 data set [34] and performance of different regression models as to evaluation metrics using this data set.Lastly, as for computational efficiency, the LM c generally requires more execution time than LM w as it generically restricts parameters.Similarly, the PM approach involves extra execution time when it applies the Box-Cox transformation to restore consistency, in other words, its execution time is dependent on the data set.Overall, in this paper we do not explore execution time in more detail, instead focusing on performance as this is the more common driver of data analysis choices in the domains considered.

VI. CONCLUSIONS
Inferential statistics and in particular (linear) regression represent a key building block of AI.For numeric variables, linear regression does not only afford a model for estimation, it provides powerful insight into the relationships between variables.Indeed, the regression-line or graph is perhaps the most widely understood visualisation in data analysis.
While well understood for the numeric case, linear regression for 'fully' IV data sets, i.e. data sets where both regressor and regressand are IV, remain challenging and an active research topic.At the same time, the potential for insights from such data sets is extremely promising, not only for regression, but for AI in general.
The complexity of IV data in terms of how intervals are distributed, how they vary in size, how they overlap-or not, makes it notoriously challenging to establish universal superiority of a single technique.Acknowledging this, in this paper, we have carried out an in-depth review and analysis of linear regression models, exploring their behaviour with respect to systematically designed synthetic and real-world, fully IV data sets, with a view to support researchers and practitioners in the choice of techniques suitable to their problem and data.
Building on existing literature, we have articulated key features of IV data, such as whether intervals are puffy, skinny, scattered, densely placed or overlapping.For all methods, we have investigated how they do, or do not, maintain mathematical coherence, and the impact of ensuring coherence on modelfitness.As part of this, we also extended current methods, proposing two variants of the LM method [20] to enhance its general suitability for real-world applications by ensuring mathematical coherence of bounds.
Going beyond the review of the state of the art and refinement of regression approaches, the paper discusses the potential of IV data in regression and AI more broadly, in particular, highlighting the value of mapping not only variable position (as in the numeric case), but also uncertainty/rangefor example to infer how uncertainty about a regressor, such as the ethical origin of a snack-food product's ingredients, impacts uncertainty-and/or position of a regressand, such as a consumer's purchase-intention.
Acknowledging the powerful aspect of visualization in terms of the popularity of traditional linear regression, we have introduced the Interval Regression Graph (IRG), facilitating the interpretation of IV regression models, and by extension, IV data.IRGs succinctly visualize and articulate the relationship between IV regressor and regressand as to both position (center) and uncertainty (range).As IRGs use all three visualizable dimensions, they are limited to the visualization of the relationship between two variables at a time.Naturally, this can be applied to multivariate cases by presenting the relationship between two variables while keeping other variables constant.An alternative to this approach could be to leverage techniques such as Principal Component Analysis (PCA) for intervals [41], and visualizing the relationships between resulting components.However, here, the intuitive interpretation of said intervals will be complex, potentially limiting one of the primary assets of leveraging intervals in real-world settings as discussed in this paper.We will explore these aspects in future work.
Finally, as alluded to above, we have leveraged the empirical results to provide recommendations on which regression models are best suited to which type of IV data sets, based on the data's features, such as whether intervals strongly overlap, are narrow or wide, etc.This is supported by complementary insights, such as generally better model prediction observed for IV sets with narrow range or condensed interval placement.Further, in this paper, we have provided open-source software which-for the first time-offers access to the suite of IV regression approaches for facilitating adoption and replication.
In summary, analyses have shown that the PM [21] and the LM [20] methods (with its weaker set of restrictions, i.e., LM w ) are the most suitable regression approaches in many cases including where intervals are skinny, puffy or a mix thereof.The constrained variant of the LM model [20], LM c also performs equally well in many cases.However, when intervals are densely placed or some overlapping/mixed, other existing methods, such as CRM [17], CCRM [18], and MinMax [16] may turn into suitable estimation approaches, offering both relatively simple implementation and efficient computation.In such instances, it is desirable to verify if the CRM [17] and MinMax [16] methods maintain consistency of the bounds.Certainly, maintaining this coherence automatically, such as by the PM, LM, and CCRM methods, tends to require more execution time.The latter may be relevant as the number of variables increases.
With increasing popularity of IV data, being able to analyse and derive the important insights available from these data using AI becomes an increasingly important topic.The substantial body of work advancing IV regression over the last ten years, and advances such as those put forward in this paper, provide the foundation for leveraging intervals more generally in AI.Substantial work remains, from further refining our capacity for communicating resulting insights to advancing our modelling techniques, such as to non-linear approaches.
Beyond advancing techniques, in the future, we aim to explore additional real-world cases to demonstrate and advance the understanding of the value of using intervals, rather than 'crisp', numeric data.Specifically, we will revisit the nature of the information captured by intervals, whether disjunctive (such as in confidence intervals), or conjunctive (true ranges, such as the real numbers between 2 and 6) and explore how different analysis techniques can, and whether they should, deal with such data differently.

Fig. 3 :
Fig.3: Relationship between regressand and regressor with respect to center (position) (a) and range (b) using the PM method for Set-1 (Fig.1(a)).compact and complete visualization, IRGs commonly combine the visualizations with respect to both the regressand's center and range as shown in Fig.18(f).We will leverage IRGs throughout the remainder with additional examples.
5, 6, and 7. Note that, Figs.3(b) and 6(b) are repeated for clarity from Figs. 1(b) and 2(b) in Section I.The IRGs in Figs.3(a) and (b) show that, according to the PM method, both changes in the estimated center and range of Set-1's regressand are associated with the changes in the center and range of the regressorin-line with our expectation.On the other hand, the IRGs in Figs.5(a) and (b) show that for the CRM method, the estimated center of Set-1's regressand is influenced only by

Fig. 5 :
Fig. 5: Relationship between regressand and regressor with respect to center (position) (a) and range (b) using the CRM method for Set-1 (Fig. 1(a)).TABLE III: Regression performance for Set-1

Fig. 7 :
Fig. 7: Relationship between regressand and regressor with respect to their center (position) (a) and range (b) using the CRM method for Set-2 (Fig. 2(a)).

Fig. 9 :
Fig. 9: IRGs showing the relationship between systolic and diastolic BP in terms of center and range for the PM method.

( a )
Ethics and overall purchase intention

Fig. 11 :
Fig.11: IRGs showing the relationship between ethics and overall purchase intention[32] in terms of center and range for the PM method.
This article has been accepted for publication in IEEE Transactions on Artificial Intelligence.This is the author's version which has not been fully edited and content may change prior to final publication.Citation information: DOI 10.1109/TAI.2023.3234930(a) Taste and overall purchase intention

Fig. 13 :
Fig.13: IRGs showing the relationship between taste and overall purchase intention[32] in terms of center and range for the PM method.

Fig. 8 ,
Fig.8, errors in model fit are fairly small in size due to densely placed intervals.Similarly, for the cyber-security data sets, Figs. 14 and 15, the PM and the LM methods show similar variability in confidence bounds, but the errors are larger as the intervals are relatively scattered, invariably impacting the feasibility of a linear model fit.Similarly, for the food snacks data with respect to ethics, Fig.10, the methods show similar variability in CI bounds, but bigger errors in terms of sizesimilar to the cyber-security data set.The only exception where the PM method shows wider variability in CI bounds than other approaches is for the taste vs overall purchase intention data set, Fig.12.This higher variability in CI bounds may be due to greater incidence of very wide and scattered intervals within the data set, resulting in increased application of the box-cox transformation and poorer fit across a larger subset of samples/simulations.Figure20in Appendix shows the total number of times across the 10000 simulations the regression approaches (CCRM, PM, LM c , and LM w ) apply the positivity restrictions/regressand bound transformations to maintain coherence for the real-world data sets.Figure20reveals that for the taste vs overall purchase intention data set, for a significant proportion (22.85% of total 10000 simulations), positivity restrictions/transformation of bounds are applied by the PM and LM w methods, compared to other sets.Exactly why the PM method is more strongly impacted than the LM w method for this case requires further research which we will address in a future publication.

Fig. 15 :
Fig.15: Graphical presentation of Evade-22 data set[34] and performance of different regression models as to evaluation metrics for this data set.

Fig. 16 :
Fig. 16: A flowchart presenting recommendations on linear regression methods with respect to different IV data sets.
This article has been accepted for publication in IEEE Transactions on Artificial Intelligence.This is the author's version which has not been fully edited and content may change prior to final publication.Citation information: DOI 10.1109/TAI.2023.3234930

Fig. 17 :
Fig.17: Comparatively low fit of the PM approach.

Fig. 18 :
Fig.18: IRGs showing the relationship between regressor and regressand in terms of their center and range using different linear regression models for Set-1 (Fig.1(a)).

Fig. 19 :
Fig.19: IRGs showing the relationship between regressor and regressand in terms of their center and range using different linear regression models for Set-2 (Fig.2(a)).

Fig. 20 :
Fig. 20: Number of times in 10000 simulations the regression approaches apply positivity restriction/transformation regressand bounds for real-world cases.BP=blood pressure data set, E&OPI=ethics vs overall purchase intention data set, T&OPI=taste vs overall purchase intention data set.

TABLE I :
Acronyms and NotationInterval {a ⊆ R : a = [a − , a + ], a − ≤ a + } a w Range of Interval, a w = |a + − a − | Y w andσ Y w Mean and Standard Deviation of Range of Y µ X w and σ X w Mean and Standard Deviation of Range of X µ Y c and σ Y c Mean and Standard Deviation of Center of Y µ X c and σ X c Mean and Standard Deviation of Center of X 3) Practical guidance on algorithm selection and software: and X + =

TABLE II :
Summary of well-known linear regression models for IV data This article has been accepted for publication in IEEE Transactions on Artificial Intelligence.This is the author's version which has not been fully edited and content may change prior to final publication.Citation information: DOI 10.1109/TAI.2023.3234930 © 2023 IEEE.Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.

TABLE III :
Regression performance for Set-1

TABLE IV :
Regression performance for Set-2

TABLE V :
Performance of different regression models for IV synthetic data sets (Set-3 to Set-11) (a) Set-3 (All disjoint skinny intervals) Methods RM SE − RM SE + M AE − M AE + M M RE The LM c , LM w , and PM methods give the best fit.The CRM and CCRM methods also closely fit the data as to M M RE.