Leveraging IT2 Input Fuzzy Sets in Non-Singleton Fuzzy Logic Systems to Dynamically Adapt to Varying Uncertainty Levels

Most real-world environments are subject to different sources of uncertainty which may vary in magnitude over time. We propose that while Type-1 (T1) Non-Singleton Fuzzy Logic System (NSFLSs) have the potential to tackle uncertainty within the input Fuzzy Sets (FSs), Type-2 (T2) input FSs provide the ability to also capture variation in uncertainty levels by means of their extra degrees of freedom. Specifically, in this paper, we propose a strategy to design Interval Type-2 (IT2) input Membership Functions (MFs) in an online manner to ensure the parameters of input MFs are updated dynamically, thus capturing varying levels of uncertainty affecting systems’ inputs. In this strategy, first, uncertainty detection is performed over a given time-frame (the Uncertainty Estimation Time-frame) and Type-1 (T1) input MFs are constructed by utilising the detected uncertainty level. Second, the variation of the uncertainty levels over a sliding window (the Uncertainty Variation Window) is used to capture the degree of variation in the detected uncertainty levels over time, which in turn informs the size of the Footprint of Uncertainty (FOU) of the IT2 MF associated with the T1 principal MF. Using time-series prediction experiments as an initial evaluation and demonstration platform for the proposed architecture, we show that the proposed strategy of designing IT2 input MFs has the potential to deliver performance benefits. Specifically, it allows systems to not only adapt to specific uncertainty levels but also to be more resilient to the variation of said uncertainty levels over time, thus offering a pathway to robust performance in real-world applications.

Abstract-Most real-world environments are subject to different sources of uncertainty which may vary in magnitude over time. We propose that while Type-1 (T1) Non-Singleton Fuzzy Logic System (NSFLSs) have the potential to tackle uncertainty within the input Fuzzy Sets (FSs), Type-2 (T2) input FSs provide the ability to also capture variation in uncertainty levels by means of their extra degrees of freedom. Specifically, in this paper, we propose a strategy to design Interval Type-2 (IT2) input Membership Functions (MFs) in an online manner to ensure the parameters of input MFs are updated dynamically, thus capturing varying levels of uncertainty affecting systems' inputs. In this strategy, first, uncertainty detection is performed over a given time-frame (the Uncertainty Estimation Time-frame) and Type-1 (T1) input MFs are constructed by utilising the detected uncertainty level. Second, the variation of the uncertainty levels over a sliding window (the Uncertainty Variation Window) is used to capture the degree of variation in the detected uncertainty levels over time, which in turn informs the size of the Footprint of Uncertainty (FOU) of the IT2 MF associated with the T1 principal MF. Using time-series prediction experiments as an initial evaluation and demonstration platform for the proposed architecture, we show that the proposed strategy of designing IT2 input MFs has the potential to deliver performance benefits. Specifically, it allows systems to not only adapt to specific uncertainty levels but also to be more resilient to the variation of said uncertainty levels over time, thus offering a pathway to robust performance in real-world applications.

I. INTRODUCTION
The real-world encompasses different noise sources and these sources may affect system inputs at different levels. While these noise sources may vary vastly and cause either major or minor impact on a system's inputs. It is in particular the variations in the noise levels which makes estimating and handling uncertainty/noise a complex and challenging task which, resulting in non-optimal outcomes. For example, attempting to remove noise from an inputs signal based on a poor noise estimation may negatively affect the system performance significantly.
Fuzzy Set (FS) theory was introduced by Zadeh [1] and is applied to Fuzzy Logic Systems (FLSs) which are considered as robust systems for capturing and handling uncertainty in decision making applications. A number of studies have shown that in cases where inputs are affected by uncertainty, Non-Singleton Fuzzy Logic Systems (NSFLSs) become more advantageous than singleton Membership Functions (MFs) designs [2]- [5]. In NSFLSs construction, each input of the FLSs is modelled as a Non-Singleton (NS) FS which enables it to capture input uncertainty.
Type-2 (T2) FSs [6] are an extension of the T1 FSs, where each degree of membership is a FS rather than a crisp number. Due to this extra degree of freedom, generally, T2 FSs provide a better ability to capture variation in uncertainty levels which is omnipresent in the real-world. [7]- [13].
Although, the NSFLS design can provide better performance in FLSs, so far, a relatively small number of studies has considered the usage of NSFLSs, including [14]- [25]. One of the reasons for this can be related to the parameter definition of Non-Singleton (NS) input MFs. Generally, the parameters of NS input MFs require either a priori knowledge of uncertainty levels or data-driven training. Yet, a priori knowledge may not be available, as the uncertainty level cannot be known in advance. Also, training procedures might be an impractical solution to define NS input MF parameters as training samples may be limited or cannot cover all the possible circumstances.
Considering the potential advantages of NSFLSs and Type-2 designs in respect to input FSs, in this paper, we propose a strategy to design Interval Type-2 (IT2) input MFs of NSFLSs at run-time. In this strategy, an uncertainty detection technique is utilised to construct T1 NS input MFs. Further, the variation of the stored uncertainty levels is used to generate a Footprint of Uncertainty (FOU) for the T1 MFs resulting in the IT2 input MFs. This ensures that IT2 input MFs are designed adaptively in an online manner which removes the requirement of a priori knowledge of uncertainty levels and training procedures in parameter definitions. Also, by modelling the variation of the uncertainty levels, the real-world uncertainty conditions affecting a system's inputs are captured in the input MFs.
The structure of this paper is as follows: Section II provides brief background information about Singleton, Non-Singleton, and IT2 FSs along with the noise detection paradigm that is used in this paper. Section III gives the proposed method to design IT2 input MFs. In Section IV, the experimental setup and associated results of the proposed approach is provided with the discussion. Lastly, in Section V, the conclusions of the current work with possible future work directions are given.

A. Singleton and Non-Singleton Fuzzy Sets
In the fuzzification step of FLSs, the system inputs are mapped to Singleton or Non-Singleton MFs. Assume that I is a fuzzy set in the universe of discourse X and the Type-1 MF I can be defined as below: where µ I (x) is the membership degree of the MF I.
The singleton MF is characterised as follows: Non-Singleton Fuzzification is useful in cases where the system inputs are affected by an external factor which causes a distortion in the actual input. To capture this distortion, inputs are designed as Non-Singleton FSs. Conceptually, in Non-Singleton fuzzification, it is commonly assumed that the crisp input x is likely to be correct value, but that because of existing uncertainty, neighbouring values of x have also potential to be the accurate values. However, the possibility of being accurate gets less and less as going further from the received x value [26], [27]. The equation of a Gaussian Nonsingleton input MFs is commonly used to model NS input MFs: where σ is the width or standard deviation of the MF which captures the uncertainty level of the NS input MF and x are the neighbouring values of the input x. An illustration of Singleton and Gaussian Non-Singleton input MFs comparison can be seen in Fig. 1.

B. Interval Type-2 Fuzzy Sets
Studies show that the ability of T1 FSs to model uncertainties and in particular variation in uncertainty may be limited when compared to T2 FSs. [28]- [30].
A T2 FS [6] is constructed using a T2 membership function formulated as 0 ≤ µ I (x, u) ≤ 1 where x ∈ X and u ∈ J x ⊆ [0, 1] as follows: Due to the high computational requirements of T2 FSs, as a special case, Interval Type-2 (IT2) FSs, was proposed and widely used for their reduced computational cost [27], [31]. In IT2 FSs designs, the secondary membership grade is set to value 1 as follows: Since IT2 FSs include an extra degree of freedom, studies have shown that FLSs which utilise IT2 FSs (other parameters such as the number of rules remaining constant) can outperform their T1 FLSs counterparts in a variety of applications [7]- [13].

C. Related Literature in NSFLSs
In the literature, a number of studies were carried out to define parameters of NS input MFs in the fuzzification in an 'offline' manner. Some preliminary works were implemented by relying on training procedures [26], [32]. Later studies were performed with the aim to achieve higher convergence speed in the training [33], [34]. These studies mainly focused on T1 parameter definitions with training procedures. However, as all the possible real-world circumstances and, in particular, variation in those circumstances cannot be known a priori, training datasets cannot fully cover each possible scenario.
Going beyond T1 input MFs, some recent applications [23]- [25] were implemented to generate IT2 input MFs. In those studies, first, a non-specified shape, convex T1 MFs is generated from a set of collected sensor values. Then, the same procedures are repeated under different circumstances to gather different T1 MFs. Lastly, by combining the gathered T1 FSs, IT2 input MF is generated by following the procedures from [35]. In [13], Quantum-behaved Particle Swarm Optimisation algorithm is utilised to define parameters of IT2 MFs of NSFLSs using offline training procedures.
Recent studies [14]- [16], [20], [21], [36] have suggested new methodologies to allow better tracking of uncertainty in the inferencing of NSFLSs. Due to the page limitation, in this paper, we focus on the traditional approach for NSFLS inference (based on using the maximum of the intersection of input and antecedent MFs). An expansion of the work and experiments to include novel NSFLS approaches will be included in a forthcoming journal publication.

D. Uncertainty Detection (Noise Estimation)
In the literature, there are numerous noise estimation techniques for uncertainty detection [37]- [41]. In the current paper, one of the initial noise estimation studies for images [42] is leveraged to estimate uncertainty levels of a frame, however we do not advocate this technique as the only or best -it was chosen as it illustrates the proposed approach well. In this estimation technique, the following calculations are performed to detect uncertainty levels: where t is the time, p is the value number in a defined window which is utilised for detection and σ n represents the noise level which is used to generate T1 input MFs.

E. Symmetric Mean Absolute Percentage Error
The symmetric mean absolute percentage error (sMAPE) is a measure of prediction accuracy of a forecasting method and it was first proposed by Armstrong [43]. Due to the sMAPE limits the error rate to 200% and is robust to outliers, it is an alternative to the commonly used Mean Square Error measure. Both sMAPE and MSE will be provided in the results section of this paper. The sMAPE measure is calculated as follows: where x t is the actual value,x t is the forecast value and N is the number of values in the time-series.

III. METHODOLOGY
In real-world circumstances, as varying levels of noise source may disturb input data at different levels/times, capturing and handling this variation of noise is an essential step for applications. Therefore, in this paper, we introduce a new strategy to capture noise variations by means of IT2 input MFs for NSFLSs.
In the methodology, first, T1 input MFs are constructed by detecting the uncertainty over a time-frame (the Uncertainty Estimation Time-frame). Simultaneously, the detected uncertainty is stored along with the previously detected levels of uncertainty. Second, the variation of the stored uncertainty levels is captured over a sliding window (the Uncertainty Variation Window) and utilised to construct an FOU which is associated with the T1 MF, resulting in an IT2 input MF.
The methodology is summarised below in six steps: 1) Time-frame for the uncertainty estimation: A sequence of observations from the input source of a system are collected over a given time-frame which is referred as Uncertainty Estimation Time-frame. Based on the design, the size/length of this time-frame can be dynamically changed or can be stable. 2) Uncertainty Detection: Over the collected observations, an uncertainty detection technique is implemented to estimate the current uncertainty level. In this step, different techniques can be utilised. For example, the algorithm shown in Section II-D can be used to compute a noise level estimate. 3) Storing the detected uncertainty: Detected uncertainty levels are stored for each Uncertainty Estimation Timeframe. Thus, as this time-frame advances, for each new time-step, a new estimate will be stored. 4) Building a Type-1 input MF: A Type-1 input MF is constructed by using the detected uncertainty level of the most recent time-frame. This detected uncertainty can be used for example with Gaussian MFs to inform their width/standard deviation.

5)
Sliding window for the stored uncertainty levels: A sliding window is defined to capture a sequence of the detected uncertainty levels. As for the Uncertainty Estimation Time-frame, the size/length of the sliding Uncertainty Variation Window can be changed according to the application. 6) Building an Interval Type-2 input MF: Here, the variation of the uncertainty levels over the Uncertainty Variation Window is computed and is used to specify the size of the IT2 FOU around the initial T1 (principal) MF generated in Step 4. The flowchart of this approach can be seen in Fig. 2. The proposed framework can be employed in a variety of applications where the input is subject to varying noise levels over time. In this paper, we focus on the time-series prediction as the generation of time-series is easily manageable and different noise levels can be added in a controlled manner.
The proposed strategy is employed by leveraging the sixstep methodology over the generated time-series as shown in Fig. 3. By following (6), the uncertainty level ( σ n ) is detected in the Uncertainty Estimation Time-frame (red dashed lines in Fig. 3) and it is stored along with the previously detected uncertainty levels. To construct Type-1 input MF, the detected uncertainty is utilised as the standard deviation of the Gaussian MF. So that the ( σ n ) is used for the σ value in (3). Thereafter, Uncertainty Variation Window (blue dashed line in Fig. 3) is defined and the variance for the estimates within this sliding window is calculated. The gathered variance is used to generate FOU of the constructed T1 input MF which resulting in IT2 input (blue MFs in Fig. 3). The well known uncertain standard deviation technique is performed by simply adding and subtracting the gathered noise variance from the actual ( σ n ) value to calculate the standard deviation of upper and lower MFs of the IT2 input MF. The proposed approach to 'envelop' (using the FOU) the variation of noise levels affecting a system over time is designed to capture the often strong variation encountered by real world systems. For example, if the environmental conditions are unstable in a robotics application, the uncertainty levels vary considerably and the adaptive FOU provides the degrees of freedom to capture this variation. If the circumstances tend to be more stable, then the variation of the uncertainty levels is smaller, resulting automatically in narrower FOUs more akin to type-1 MFs, enabling the system to respond quickly and efficiently to changes in its inputs. Fig. 4 illustrates this adaptive behaviour. Even though the noise level estimated most recently is similar in both examples, the variation of the noise over the window (note, here, both the windows are of equal size for simplicity) differs substantially in both cases. Specifically, in Fig. 4a, the noise levels are relatively stable over the time ( σ n ), in Fig. 4b, the noise levels vary significantly. The adaptive input MF and FOU generation thus leads to two very different IT2 input MFs, with a smaller FOU generated for the more stable case on the left (Fig. 4a) and a wider FOU for the more unstable case on the right (Fig. 4b).

IV. EXPERIMENTS AND RESULTS
As one of the commonly used chaotic time-series, Mackey-Glass (MG) is used to implement a series of forecasting experiments in this paper. In order to provide chaotic behaviour in MG, τ value is set to 30, while a = 0.2 and b = 0.1. During the time-series generation, firstly, 2000 samples (from t=−999 to t=1000) are generated and due to the fluctuation tendency in the initial part of the time-series, the last 1000 (from t=1 to t=1000) values are taken to be used in the experiments.
For the training phase of the Mamdani [44] NSFLSs, rule generation is completed by utilising the common Wang-Mendel method [45] over the first 70% of the data points from the noise-free MG series. Nine past points are used to make the prediction and seven antecedents are utilised in the rule generation. For the testing phase, two different noisy data sets are generated. In the inference step of NSFLSs, the minimum t-norm AND, max t-conorm OR operators are utilised and the same number of discretisations (500) are used for all NSFLSs. In order to mitigate the effect of randomness in the noise addition process, each experiment is repeated 30 times, while the prediction error is captured using both the commonly used Mean Square Error (MSE) and the average of the generated Symmetric Mean Absolute Percent Error (sMAPE) measures. Note that, in this paper, we are not interested in achieving the best possible prediction accuracy, but in the relative differences in performances between different NSFLSs.
Two different set of experiments, with different testing timeseries data sets, are implemented to conduct the comparison between the proposed adaptive and non-adaptive NSFLSs. As nine past points are used in the prediction, the frame and the window sizes (Uncertainty Estimation Time-frame and Uncertainty Variation Window) are set to nine as well in this paper. While this is an intuitive choice, other choices may provide superior results -this may be investigated as part of future work.
In both experiments, we investigate that which FOU value in IT2 input MFs provides the least prediction error and how does this compare to the proposed adaptive approach. In the investigation, 10 different sequential FOU values, between 0.01 and 0.1, are picked and applied to IT2 input MFs in each set of experiment. Then the proposed adaptive approach is implemented to generate FOU in an online manner and the results are compared to manually adjusted FOU experiments.

A. Experiment A -FOU creation under stable noise levels
In this experiment, the noise level in the test time-series is kept constant to evaluate two main points: (i) Demonstrate and analyse the behaviour of the proposed adaptive approach in terms of the generated FOU sizes under stable noise. (ii) Compare the adaptive prediction results against the nonadaptive different NSFLSs with manually adjusted FOU sizes in order to explore how arbitrarily selected FOUs affect the FLSs performance in comparison to those arising from the adaptive approach. The experiment is performed by following the steps below: 1) SNR 20 dB Gaussian noise is added to each value of the noise-free testing set. 2) Next, the proposed adaptive IT2 input MF generation strategy is implemented and prediction results captured using both the MSE and sMAPE measures. 3) After evaluating the adaptive approach, a follow-on set of experiments is conducted where a set of predefined FOU sizes are used, rather than automatically adapting order to validate if the adaptive approach successfully generates suitable FOU sizes in this most simple casewhen uncertainty is constant, rather than varying. An illustration of the used time-series is provided in Fig. 5.

B. Experiment B -FOU creation under varying noise levels
Since the proposed technique is designed to capture the varying noise levels common in real-world settings, in this experiment, different noise levels are sequentially injected into the time-series test set. Specifically, a sequence of relatively low (20 dB) and high level (0 dB) of noise is injected as illustrated in Fig. 6. While the beginning of this time-series is subject to a low, 20 dB noise level, this is increased to a high 0 dB level and after that, reduced back to the low 20 dB level of noise (see the middle part of Fig. 6). By generating this pattern of noise variation, we attempt to replicate real-world situations where an unexpected disturbance suddenly occurs in the signal data (e.g. light variation affecting a camera). Five examples of the generated IT2 input MFs are shown in the lower part of Fig. 6. The same steps from the Experiment A are followed for performance evaluation.

C. Results
The results of Experiment A in Fig. 7 show that the adaptive approach and the best manually specified FOU have similar sMAPE and MSE results which indicates an accurate adaptation of the FOU size in this context where the noise remains constant. In a real world context, it would of course not possible to run all different FOU sizes sequentially, and the proposed approach for its automatic adaption would be strongly preferable.
In Experiment B, the results visualised in Fig. 8 reveal that the adaptive strategy again produces both sMAPE and MSE results which are close to the optimal manually selected FOU size. We note that the performance of the adaptive approach is not substantially better than any manually selected FOU size even though the uncertainty levels are varied across the testing set. At the time of writing this article, we believe that this may be due to the overall limited level of variation of noise in the testing set, i.e. while the noise level is changed twice across the time series, this limited amount of variation can be captured well by predefined FOU sizes, thus do not leave much scope for the adaptive approach to show improved performance.

D. Discussion
In this paper, two different noisy time-series datasets (stable and varying noise) are used in the testing of the NSFLSs and the proposed adaptive strategy is utilised to generate IT2 input MFs with uncertain standard deviation and different FOU values are used to make the comparison.
In Experiment A, the constant level of noise is used in the testing time-series data set in order to analyse the behaviour of the proposed adaptive approach. During this experiment, the noise level remains constant at 20 dB, resulting -as expected -in a constant FOU size as shown in Fig. 7. In Experiment B, the varying noise levels result in adaptation of the FOU sizes. When the data exhibits low variation in the levels of noise, the FOU is smaller, while high variation results in larger FOUs. We note in particular the narrower FOU size towards the middle of the time series (middle MF in Fig. 8). It reflects that the noise level is constant (at 0 dB) toward the middle of the time series (at step 870), while the MFs are points of the time series where the noise levels are changing (e.g. 800 and 910) show wider FOUs.
Further, as shown in Figs. 7 and 8, the proposed adaptive approach produce MSE and sMAPE values which are close to the lowest error values achieved by the best of the 10 NSFLSs with fixed, manually specified FOU sizes. Although it may seem fixed FOU sizes are used (i.e. 0.01 or 0.02) and there are no substantial differences between the adaptive and the best manually specified NSFLS results, it is crucial to note that when we use the proposed adaptive technique, we do not need to know any a priori information about the noise levels, their variation or any specific FOU sizes. I.e. in real applications, manually adjusting FOU sizes of IT2 input MFs would both be impractical and commonly not feasible as no information is available on what FOU size would result in the best performance for future levels of noise variation.

V. CONCLUSIONS
In this paper, a new type of noise-robust, adaptive IT2 input NSFLS is proposed which enables on-the-fly adaptation of the input fuzzy set to capture both different levels of noise Fig. 8. Experiment B -The MSE and sMAPE prediction result comparison of adaptive and non-adaptive FOU generation in the test set with varying noise levels (affecting the input(s)) and variation in said levels. Specifically, principal T1 input MFs are generated by detecting uncertainty within an Uncertainty Estimation Time-frame. Then, the T1 principal MF is extended with an FOU to an IT2 input MF, where the size of the FOU is driven by variation of uncertainty levels as measured over an Uncertainty Variation Window and the FOU is applied to T1 input MFs.
The proposed adaptive approach allows for the construction IT2 input MFs dynamically in a fully online manner, which can capture and handle both noise and changes in noise levels without requiring any a priori information on noise levels.
Initial experiments conducted in this paper based on MG time series prediction show that the proposed adaptive strategy provides a promising approach to dealing with noise in realworld applications.
In the future, the proposed approach will be further evaluated both with other time-series datasets and other applications (such as in robotics). Further, we will specifically focus on evaluating the architecture's performance under conditions where noise levels vary strongly (e.g. Quadcopters subjected to wind gusts). Finally, we will explore the integration of recently proposed advanced techniques [14], [15], [21] to determine the firing strength of NSFLSs which are designed to model the interplay of input and antecedent MFs (and associated uncertainty models) with high fidelity. Also, Type-2 and Type-1 comparison will be implemented by varying window sizes in an online manner.