Time series forecasting with interval type-2 intuitionistic fuzzy logic systems

Conventional fuzzy time series approaches make use of type-1 or type-2 fuzzy models. Type-1 models with one index (membership grade) cannot fully handle the level of uncertainty inherent in many real world applications. The type-2 models with upper and lower membership functions do handle uncertainties in many applications better than its type-1 counterparts. This study proposes the use of interval type-2 intuitionistic fuzzy logic system of Takagi-Sugeno-Kang (IT2IFLS-TSK) fuzzy inference that utilises more parameters than type-2 fuzzy models in time series forecasting. The IT2IFLS utilises more indexes namely upper and lower non-membership functions. These additional parameters of IT2IFLS serve to refine the fuzzy relationships obtained from type-2 fuzzy models and ultimately improve the forecasting performance. Evaluation is made on the proposed system using three real world benchmark time series problems namely: Santa Fe, tree ring and Canadian lynx datasets. The empirical analyses show improvements of prediction of IT2IFLS over other approaches on these datasets.


I. INTRODUCTION
T IME series forecasting is an important application area that has been extensively researched. It involves the sequential collection of observations over time with the purpose of developing a model that captures the underlying dependencies among attributes of the data. A wide range of approaches have been employed in the analysis of time series data. More recently, the use of soft computing methodologies such as fuzzy logic (type-1 and type-2), neural networks, simulated annealing and genetic algorithms have been reported in the literature for time series forecasting [1]- [4]. These latter approaches have shown significant improvements over the traditional statistical methods because they are non-linear and are able to approximate any complex dynamical systems better than linear statistical models [5].
Due to the prevalent uncertainty in data, fuzzy logic models have been widely adopted. The reason behind this is that the general framework of fuzzy logic connotes uncertainty. However, because fuzzy logic systems lack the learning capability, they are often hybridised with learning algorithms such as artificial neural networks (ANNs) -an approach that is also adopted in this study. Fuzzy set (FS) theory was introduced by Zadeh [6] as a generalisation of the classical notion of a set. From the literature, a type-1 fuzzy logic system (T1FLS) models uncertainty to a certain degree in many applications and may not handle or minimise the effects of uncertainties in some real world applications [7]. Zadeh [8] therefore extended type-1 fuzzy set (T1FS) to include type-2 fuzzy set (T2FS) which can handle uncertainties in a better way in many applications because membership grades of T2FS are themselves fuzzy.
Atanassov [9] extended the concept of Zadeh's fuzzy sets to intuitionistic fuzzy sets (IFSs), which handle uncertainty using both the degrees of membership and non-membership of an element x to a fuzzy set A together with some degree of hesitancy. Intuitionistic fuzzy sets of type-1 have been successfully employed in time series forecasting [10], [11]. However, because type-1 intuitionistic fuzzy sets (T1IFS) have membership and non-membership grades that are precise, they may not handle uncertainty well in many applications (see [12]).
To this end, we propose an application of the recently developed interval type-2 intuitionistic fuzzy logic system (IT2IFLS) [12] framework for time series analysis. The IT2IFLS utilises extra degrees of freedom in terms of interval non-membership functions. This constitute a point of departure of our model from other existing classical fuzzy models. Our motivation is to apply IT2IFS to model uncertainty in time series data with the objective of obtaining the minimum prediction error.
The rest of the paper is structured as follows: In section II, IFS, T2IFS and IT2IFS are defined. In section III, IT2IFLS is designed and parameter update rules are derived. We present our results in Section IV, and conclude in section V.
[9] Given X as a non-empty set, an intuitionistic fuzzy set A * in X can be represented as: is the membership grade and ν A * (x) : X → [0, 1] is the nonmembership grade of element x ∈ X and for each x ∈ X, When ν A * (x) = 1−µ A * (x) for all x ∈ X, then A is a fuzzy set. Moreover, π A * (x) = 1 − (µ A * (x) + ν A * (x)) defines the hesitancy of elements x in A * .
Thus, when uncertainties disappear, a type-2 membership and non-membership functions must collapse to a type-1 membership and nonmembership functions respectively and their amplitudes must where represents union over all admissible values of x and u for a continuous universe of discourse, and is used instead for discrete universe of discourse. When µÃ * (x, u) = 1 and νÃ * (x, u) = 1, a simpler version (IT2IFS) of a T2IFS is obtained. (see Figure 1 and Equation (1)).
where x is the primary variable, and u is the secondary variable. The uncertainty about an IT2IFS is completely described by membership and non-membership footprints of uncertainty (FOUs) (see Figure (1)) and defined as in Eqns (2) and (3) [12]

III. INTERVAL TYPE-2 INTUITIONISTIC FUZZY LOGIC SYSTEM
Similar to a type-2 fuzzy logic system (T2FLS), a T2IFLS consists of the fuzzifier, rule base, fuzzy inference engine and output processing module.

A. Fuzzification
The fuzzification process employed is the interval singleton type-2 fuzzification and involves the mapping of a numeric input vector x ∈ X into an IT2IFSÃ * in X which activates the inference engine. Here, interval singleton type-2 fuzzification is employed. Hence, for every input where π c,ik is the IF-index of center and π var,ik is the IFindex of variance [14]. The antecedent parameters areσ 2,ik , σ 1,ik , π c,ik and π var,ik .

B. Rules
The IF-THEN rule of an IT2IFLS has the same syntax as that of the classical IT2FLS. The only difference is that IT2IFS are used in the rule formation instead of ordinary IT2FS. The rule representatipn is as follows: R k : IF x 1 isÃ * 1k and x 2 isÃ * 2k and · · · and x n isÃ * nk T HEN y k is f (x 1 , x 2 , · · · , x n ) = w 1k x 1 + w 2k x 2 + · · · + w nk x n + b k (4) whereÃ * 1k ,Ã * 2k , · · · ,Ã * ik ,· · · ,Ã * nk are IT2IFS and y k is the output of the kth rule formed by linear combination of the input vector: (x 1 , x 2 , · · · , x n ).

C. Inference
The IT2IFLS proposed in this study benefits from type-2 intuitionistic membership and non-membership functions in the antecedent parts and a linear function in the consequent parts, otherwise referred to as A2-C0 intuitionistic fuzzy system The output of IT2IFLS is computed as follows [12]: µ k , f ν k and f ν k are the lower membership, upper membership, lower non-membership and upper nonmembership firing strength respectively. The implication operator adopted for the study is the "prod" t-norm such that: where * is the "prod" operator, y µ k and y ν k are the output of the kth rule corresponding to membership and non-membership functions respectively. The user defined parameter β specifies the contribution of the membership and non-membership values in the final output, where β ∈ [0, 1]. It follows that if β = 0, only the membership function is involved in the output generation and if β = 1, then the system's outputs is a function of only the non-membership. The parameters of the antecedent and consequent parts of the rules are tuned using gradient descent algorithm (GDA). The cost function for a single output is defined as follows: where y a is the actual output and y is the simulated output.
The parameter update rules are as follows: where γ is the learning rate(step size). The derivatives in Equations (6) and (7)  In the next section, we carry out the evaluation of the developed model to assess its efficiency and performance on three real world application domains.
IV. EXPERIMENTS AND RESULTS In this section, three real world applications are investigated to verify the feasibility and effectiveness of the proposed IT2IFLS on time series datasets namely Santa Fe, tree ring and Canadian lynx time series. The experiments are conducted and the results are compared with similar studies in the literature. The performance criteria used for the experiments are the root mean square error (RMSE), the non-dimensional error index (NDEI) and the mean absolute error (MAE) as expressed in Equations (11) to (13) respectively.
where y a is the desired output and y is the output of the model, N is the number of testing data points. Two intuitionistic Gaussian membership and nonmembership functions are utilised for all experiments. The value of β is initially set to 0.5 to allow equal initial contribution of membership and non-membership functions to the model outputs. The values for the weights (w) and bias (b) are randomly generated in the interval [0,1]. All experiments are carried out using M AT LAB c 2014 running on a 64-bit Intel core i3-4130 CPU@3.40GHz /8GB RAM configuration computer.

A. Example 1 -Santa Fe time series
This example considers the Santa Fe Laser dataset of the Santa Fe time series competition obtained from [15]. The data were measured from a far-infra-red laser in a chaotic state. This series was earlier analysed in [16], and a model called pattern modeling and recognition system (PMRS) was proposed. The performance comparison using neural network (NN) and a statistical exponential-smoothing (ES) was also reported. The Santa Fe time series is a univariate time series measured from a physical system in the laboratory. The computational protocols are similar to [17], 1000 samples are used with five inputs and one output with the data generation vector represented as: y(t) = (y(t−1)y(t−2)y(t−3)y(t−4)y(t−5)). All samples are scaled to within the range [0, 1] by dividing each by the maximum value of the dataset. Similar to [17], 90% of the samples are used for training while 10% are used for testing. The parameters β = α = 0.5 with 100 training epochs. Table I   The tree ring time series obtained from [21] contains annual measures of tree rings width measured in Argentina for the period 441-1974. The dataset is randomly split into 75% training and 25% testing. In [22], the evolving fuzzy optimally pruned extreme learning machine (eF-OP-ELM), was reported for analysing this time series. The dynamic evolving neuro-fuzzy inference system (DENFIS), evolving Takagi-Sugeno (eTS) model and online sequential method for fuzzy systems based on online sequential ELM (OS-fuzzy-ELM) were also reported in [22] for tree ring time series analysis. All computational protocols in this study are arranged as close as possible to those reported in [22] to ease comparison.   [23] 0.959 0.624 eTS [24] 0.714 0.457 OS-Fuzzy-ELM [25] 0.794 0.511 eF-OP-ELM [22] 0.841 0.536 IT2IFLS-TSK 0.395 0.157

C. Canadian Lynx Time series
This time series is selected in order to compare the performance of IT2IFLS on non-fuzzy approaches. Canadian lynx  [26]- [28], the logarithms to the base 10 of the data are used in the analysis. Figures 4 and 5 show the original and the logarithmic transformed data of the Canadian lynx series respectively, with a periodicity of approximately 10 years. The series consists of 114 observations of which 100 samples are used for training and the remaining 14 are used for testing in order to validate the effectiveness of the model proposed in this study. Similar to [28], the maximum training epoch adopted is 2000. As shown in Table III, IT2IFLS outperforms the listed non-fuzzy approaches on the Canadian lynx dataset.   [26] 0.020486 0.112255 ANN [26] 0.020466 0.112109 ANN (p,d,q) [29] 0.013609 0.089625 Zhang's Hybrid ARIMA/ANNs model [26] 0.017233 0.103972 Hybrid ARIMA/ERNN model [30] 0.009 -SETAR [31] 0.014 -FNN [31] 0.009 -Generalised Hybrid ARIMA/ANNs model [32] 0.00999 0.085055 ANN/PNN model [27] 0.014872 0.079628 ARIMA/PNN model [27] 0.011461 0.084381 MNM-ANN-DEA [33] 0.

V. CONCLUSION
In this study, an IT2IFLS-TSK approach to time series forecasting is presented. The IT2IFLS can accommodate more imprecision in terms of non-membership function, thereby modelling imperfect and imprecise knowledge better than some classical fuzzy approaches. The key contribution in this study is the integration of non-membership function and IF-index in IT2FLS (IT2IFLS). These extra parameters of IT2IFLS give it additional design degrees of freedom to handle uncertainties well. In future, we intend to learn the parameters of the IT2IFLS using hybrid approach of GD and Kalman filter algorithms. We also intend to apply IFS to general T2FLS.