An investigation into the relationship between type-2 FOU size and environmental uncertainty in robotic control

It has been suggested that, when faced with large amounts of uncertainty in situations of automated control, type-2 fuzzy logic based controllers will out-perform the simpler type-1 varieties due to the latter lacking the flexibility to adapt accordingly. This paper aims to investigate this problem in detail in order to analyse when a type-2 controller will improve upon type-1 performance. A robotic sailing boat is subjected to several experiments in which the uncertainty and difficulty of the sailing problem is increased in order to observe the effects on measured performance. Improved performance is observed but not in every case. The size of the FOU is shown to be have a large effect on performance with potentially severe performance penalties for incorrectly sized footprints.


I. INTRODUCTION
A fuzzy logic system maps single discrete inputs into a fuzzy set by means of a fuzzifier, the fuzzy set output can then be processed as part of a Fuzzy Inference System (FIS) where the set is used as an input to an inference system with its associated rule base. This results in a new output set that can itself be defuzzified into a discrete value suitable for use as a standard (e.g. PID) controller output.
The exact nature of the processing and the fuzzy sets obtained can be varied based upon application specific requirements and restrictions and this gives rise to three main subcategories of fuzzy control: type-1, interval type-2 and general type-2. In this paper we consider type-1 and interval type-2 based systems and intend to apply both to control problems of increasing complexity with differing quantities of uncertainty and noise thereby hoping to develop a method by which an optimal controller parameters for a given problem can be determined as well as the possibility for criteria to select which variety is the most appropriate for a given situation.
The application discussed in this paper is one of robotic sailing using the FLOATs (Fuzzy Logic Operated Au-Tonomous Sailboat) platform further described in [2] in which a robotic sail boat actuates sail and rudder positions based on received sensor data with the aim of steering an autonomous boat around a predetermined course. Similar boat based applications have been investigated with a variety of techniques such as PID [8], biologically inspired [10] and fuzzy methods as in [1] and [12]. We have selected this application as it offers multiple sources of noise and uncertainty in the environment which is useful for discerning where type-2 and in general more sophisticated controllers will out perform other more simple type-1 varieties.
This paper is organised as follows: Section II provides background and information into the methods, software and systems used in the rest of the paper. Section III describes the experiments that will be performed and will be followed by section IV where we state our numerical results and section V where these results are discussed. Finally in section VI we draw some conclusions along with some discussion about future work.

A. Type-1 Fuzzy Logic
Type-1 fuzzy logic, introduced by Zadeh in [17] uses 2D membership functions that are commonly triangular, Gaussian or polygonal in shape. The x axis represents the set of possible inputs into the system and the y axis represents the degree to which the given input is part of the fuzzy set and may have a value between 0 (no membership to the set) and 1 (complete membership to the set). The membership functions that will be used for the type-1 controller in the paper is shown in figure 1(a).
One of the main perceived shortcomings of type-1 fuzzy logic is that it provides no obvious mechanism to model uncertainty that may be present in an environment and adapt to it without changing the controller. It has been said by Mendel in [5] that this shortcoming means that a type-2 controller, with its footprint of uncertainty should be able to improve performance on a type-1 in situations of high noise. However the exact meaning of a "high noise environment" has as yet not been defined.

B. Type-2 Fuzzy Logic
Type-2 fuzzy logic is a development and evolution of type-1 as described in previously in section II-A however, with this development comes greater freedom and flexibility as discussed by Wu in [15] which may increase performance in certain situations such as explored by Hagras in [3] at the potential cost of increased computational load. Much of the increased flexibility of this type of system derives from the fact that the membership functions in type-2 systems are represented in three dimensions instead of type-1's two. This extra dimension gives rise to what is known as the secondary membership function and its usage determines the exact variety of type-2 that is being used.
In an interval type-2 system the value of the secondary function is limited to either zero and one, allowing interval type-2 systems to be represented by two individual membership functions in the 2D plane, termed the upper and lower membership functions (UMF and LMF respectivly), they the area that they enclose termed the footprint of uncertainty (FOU).
With general type-2 however any mathematical function that gives outputs between 0 and 1 can be used making it the most complex of those described here, for a more comprehensive overview of type-2 fuzzy logic the reader is directed at a paper such as [6] or [4].
Another significant difference between type-1 and type-2 fuzzy logic aside from the fuzzy sets themselves is the extra step that is needed in type-2 fuzzy processing -that of typereduction. As a type-2 inference system will return at least 1 type-2 fuzzy set, which cannot be directly defuzzified, an additional process known as type-reduction must occur in order to as the name suggests, reduce this type-2 set into type-1 which can then be defuzzified into a usable output.
While both interval and general type-2 are more complex than type-1 due to the extra dimensionality of the sets and the additional processing required, methods to reduce complexity for processing are being developed such as by Wu in [16] and Wagner and Hagras [14] both of which aim to reduce the computational load significantly into realms manageable by resource constrained systems such as those used on mobile robots.
In our application we derive our type-2 footprints of uncertainty by introducing a horizontal movement to the type-1 membership functions with the amount of movement being used will be varied as a parameter value. Figures 1(b) and 1(c) show examples of this with the type-1 membership function being moved 10 • and 20 • respectively to give the shown FOUs.

A. Problem description
The application to be used as our test case is FLOATS described in [2]. It is an autonomous sailing boat originally based on the work done by Stelzer in [13] which can work in both simulation and the real world environments. Wind, location, waypoint location and direction sensors are used to calculate error and delta error values which are used as inputs to a fuzzy inference system which produces a rudder change output value.
In this paper we have opted to use simulation as the method of experimentation with real life experimentation planned for future work. The simulator used, Tracksail has been used by others for development and testing of autonomous sailing robotic systems by Sauze and Neal [9]. Tracksail is Java based and communicates with controllers by means of a standard network socket allowing virtually any implementation language to be used for the controller.
A running rate of 1Hz was fixed in the controller code for all controller configurations, this value was chosen in order to ensure that the more sophisticated controllers could run a complete cycle as there were initial concerns that for type-2 based controllers especially the overhead would be too high for a faster rate. While this low running rate will lead to overall lower performance as all controllers are subject to the same restrictions we believe the comparison to be fair.
The cumulative RMSE (Root-mean, square error) between the current and desired heading (measured in degrees) will be the main metric used for comparison. As is usual with control experiments lower values represent a better performing controller.

B. Hypothesis
We hypothesize that there will be a point at which complexity of the course is sufficient that the type-2 will significantly improve upon the results obtained by the standard type-1 controller. We expect that this point will occur more obviously in situations with a higher uncertainty score as described in table II and with the extra turns exaggerating the effect further.
It is expected that as the various wind configurations are tested the RMSE will increase in a predictable order -with early configurations such as A and B showing lower RMSE values than the later configurations such as H and I. We do not expect a straight order increase especially as several configurations have the same uncertainty score.
We also expect to see that as the FOU size of the type-2 based controllers is increased the performance will start at type-1 levels as a 0 size FOU is equivalent to a type-1, followed by a increase in performance and then a drop off in performance as the FOU increases to cover the entire universe of discourse resulting in extremely bad performance which in the worst cases will prevent the course from being completed at all.

C. Experimental Design
The controllers under test will maintain the same membership functions throughout all experiments, with the type-1 membership functions being shown in 1(a) and the only change being the horizontal movement of the type-2 controller which alters the size of the FOU. We will test six different values for this FOU size, starting at 0 and increasing to a maximum of 25 in 5 increments with an FOU with 10 being shown in figure 1 We intend to use two separate mechanisms to gradually increase complexity of the environment to in order to highlight the differences in performance that can be achieved with the various fuzzy systems under test. The first mechanism is the way point system of the simulator which will allow us to define the number and degree of turn that the controller must steer the boat through in order to complete the course. The second mechanism is the introduction of noise into the environment in the form of variations of the wind, table I outlines the configurations of wind that we will be using in this experiment.
An automated control rig will be used to execute a batch of 30 separate runs for each configuration of controller, parameter value (i.e. FOU size) and course layout with each piece of software (controller, simulator and common code) maintaining its own logging files that can be analysed to produce RMSE values.

D. Course Design
The courses will be built up from the simplest of all courses -a straight line with a parallel fixed wind in which the boat must simply move forward in order to complete. The complexity will then be increased by adding turns of varying angles as shown in figure 2, where it can be observed that we will uses courses requiring either one or two turns. The vertical movement required to complete the course, defined as either 0, 25, 50 or 100 'metres' will define the angle that the boat must turn in order to complete the course. For each of these course layouts every wind configurations, as shown in table II, will be applied, starting with no noise (configuration A) to a high noise environment (Configuration I), each change occurring at four second intervals. This gradual increase of noise will allow structured observations to be made about the effects of noise upon the performance of type-2 controllers with varying FOU sizes.

IV. RESULTS
One sided Wilcox tests were used to test the statistical significance of the difference between two individual batches of experiments. The result for this test is a P-Value with a small value (¡0.05) indicating a statistical significant difference.
The first test for all experiment configurations was a  We first consider figure IV which shows the results of a benchmark experiment in which the majority of controllers simply maintain a straight course when the parameter value was set to 0 (equivalent to type-1), giving the expected aver-  figure 3(a). We believe that these results are caused by the fact that the controller does not need to execute any turns or course corrections in order to complete the task meaning any performance benefits/penalties a controller may have when turning do not have a chance to become apparent. The next set of data to be considered is shown in figures 4 and 5 which show how the RMSE value (on the y-axis) change as the FOU size is increased from 0 to 25 (as shown on the x-axis) for each wind configuration (and coloured line) with each course configuration shown in a separate figure. In each of the figures we can see some similarities such as in general the RMSE increases (signifying decreasing performance) as FOU size exceeds 20 and in many cases the decrease in performance is very large indeed. We can also observe that improvements in performance happen directly before this point, usually at 20 -this is most obvious in Figure 4(b), but can also be observed in others such as Figure 4(a) and Figure 5(b). Figures 6 and 7 plot the difference between type-1 controller and the best performing type-2 controller which as stated previously usually occurs when the FOU is 20. The xaxis shows the different wind configurations under test while the y-axis shows difference in RMSE between the type-1 and type-2 controllers. The data for vertical movement 50 and 100 with a single turn are shown in tables III and IV. It is hard to observe any obvious patterns in these plots The figures 8 and 9 show example course plots of single and two turn courses with all the various wind configurations under test represented by coloured lines and the white circles indicating way points that must be reached to complete the course. We can see a rise in complexity of the course with both angle and number of required turns increasing, from left to right, which in turn seems to be leading to more runs showing additional turns such as green line in Figure 9(c) being a good example.
Tables V and VI show the P-Values obtained when comparing the type-1 controller and the best performing FOU size for each wind configuration and vertical movement combination for one and two turn experiments respectively. If there is no value that is lower than the type-1 value then this configuration is omitted from the table. We can observe here firstly that there are no points in which the vertical movement is 100 and secondly that two turn experiments have considerably fewer points than the single turn. The reasoning and understanding for this will be discussed in the next section.
V. DISCUSSION It can be observed from the results obtained and outlined in the previous section that type-2 based controllers can and do out-perform type-1 controllers in several circumstances but that this does not occur in the majority of cases and it is in fact more common for the performance to be similar to the type-1 value (statistically so in many but not all of cases). If we enumerate the number of cases we find only 23 of that total 324 (Comprised of nine Wind configurations, six different FOU sizes, three different vertical movement values and two different turn counts) show statistical improvement which equates to approximately 8%. This relativity low percentage shows that those who move from type-1 to type-2 expecting a large increase in performance essentially for free with their applications are more than likely to see at best the same performance but in most cases significantly worse unless considerable design effort is undertaken.
Our results are supported by other works in which type-2 performance is compared with type-1 such as the work    figure 10 in which the RMSE for each wind configuration in each vertical movement and turn count combination is plotted with the FOU size being held at 20. In the majority of cases wind configuration 'B' (red points) tends to have one of the the highest RMSE over the entire range of FOU sizes. This contrasts with wind configuration 'I' (orange points), which seem to often appear at the bottom of the graph indicating the best performance. This seems counter to what would be expected which would be for wind configuration A to have the lowest RMSE and configuration I to have the highest as common sense would dictate that more noisy environments are more difficult to sail in. Whether this conclusion is a general result or an artefact of the nature of this control problem is not yet known but will be the subject of future work. We also observe the spread of the results for different wind configurations increases with the course complexity. When the vertical movement is 25 with a single turn (the simplest non 0 experiment) the results are much closer together with a difference between highest and lowers RMSE value a type-1 controller of 0.65 which contrasts significantly with the 100 two turn experiment (the most complex) in which the difference is 8.0. This is an expected result as with each increase in course complexity the amount of work that must be done by each controller increases meaning there is greater scope for a controller to demonstrate its improved performance compared to a less well performing one.
The Figures 6 and 7 show that there is no obvious correlation between wind configuration (and therefore environmental noise) and the performance change achieved when moving to a type-2 controller. This could be down to the ordering of the configurations as defined in table II as multiple configurations have been given an equal total uncertainty score due to the assumed equal weighting of the two noise sources which may be a false assumption as discussed previously. This also contrasts with the findings made by Sepulveda et al [11] in which type-1 and type-2 controllers are tested and the type-2 out performs the type-1 in all cases both with and without uncertainty and the difference in performance seems to have an increasing correlation which suggests either the difference is down to application or that they have simply not tried as many noise configurations as we have done here which has not allowed the differences we have found here to present themselves.
The addition of turns to increase the complexity of the course has a significant effect on the performance of the controllers. It can be observed between figures 4 and 5 that every value is higher in the two turn situation in comparison with the single turn. One of the reasons behind this is the angle that must be turned in the second turn is in fact larger than the first turn by an approximate factor of 2 -this will need to be analysed prior to any similar work being carried out.

VI. CONCLUSIONS & FUTURE WORK
It can be observed from the results obtained and outlined in the previous section that type-2 based controllers can and do out-perform type-1 controllers. However care must be taken in the design of the type-2 system -especially with regards to the size of the FOU -too small and it will not improve over the type-1 but too large and it will perform worse. In our selected application the value of 20 seems to be the optimal value. Further work will be required to determine the reason for this value.
Overall this work shows the association between performance change and environmental noise to be considerably more complex than previously assumed. The statement that increasing environmental noise will lead to the type-2 performance compared to type-1 performance increasing in magnitude is not supported by the results in this paper.

A. Future Work
The next step in this work will be to perform these experiments in a real world environment and to observe to what degree the findings in here in simulation apply a real world control problem.
Additional simulation work will also be required including increasing the FOU beyond the shown done here and also applying a more gradual increase (e.g. By 1 increments instead of 5) as it would be of interest to observe how sensitive the optimal point is.