, Allison (2016) Multisensor navigation systems: a remedy for GNSS vulnerabilities? Proceedings of the IEEE, 104 (6). pp. 1339-1353.

 Abstract —Space-based positioning, navigation and timing (PNT) technologies, such as the Global Navigation Satellite Systems (GNSS) provide position, velocity, and timing information to an unlimited number of users around the world. In recent years PNT information has become increasingly critical to the security, safety and prosperity of the World’s population, and is now widely recognized as an essential element of the global information infrastructure. Due to its vulnerabilities and line-of-sight requirements GNSS alone is unable to provide PNT with the required levels of integrity, accuracy, continuity and reliability. A multi-sensor navigation approach offers an effective augmentation in GNSS-challenged environments that holds a promise of delivering robust and resilient PNT. Traditionally, sensors such as inertial measurement units (IMUs), barometers, magnetometers, odometers and digital compasses, have been used. However, recent trends have largely focused on image-based, terrain-based and collaborative navigation to recover the user location. This paper offers a review of the technological advances that have taken place in PNT over the last two decades, and discusses various hybridizations of multi-sensory systems, building upon the fundamental GNSS/IMU integration. The most important conclusion of this study is that in order to meet the challenging goals of delivering continuous, accurate and robust PNT to the ever-growing numbers of users, the hybridization of a suite of different PNT solutions is required.

Over the past two decades, Global Navigation Satellite System (GNSS) technology has proliferated into almost all infrastructure and consumer segments of developed economies and societies. Transport systems, mobile communications, power grid networks and financial systems are examples of critical infrastructure that are heavily dependent on GNSS. In a similar way, the mining, agriculture and construction sectors have developed a strong dependence on the availability of trustworthy GNSS information for automated machinery operations to drive advances in industry productivity. Additionally, millions of people trust positions from GNSS for everyday needs, such as routine navigation and route guidance. Consequently, the requirement for a positioning capability that is truly reliable and robust, i.e. operational everywhere and anytime, that is trustworthy with an accuracy fit for purpose, has become increasingly significant. Although GNSS vulnerabilities are well understood, this increasing ubiquity of GNSS across society has heightened our awareness to the risks associated with performance anomalies. The challenge has, therefore, become one of delivering the GNSS-like performance levels we have come to expect -under ideal operating conditions -in all environments, but most importantly in densely built-up areas, indoors and underground. To address this challenge, this paper reviews some of the approaches that can be used to detect and remedy irregularities or deviations from normal GNSS operations. To illustrate the variety of environments, dynamics and potential Positioning, Navigation and Timing (PNT) sensors that could be experienced and used during a simple journey this paper will start with a typical example scenario. This is followed by a description of multi-constellation GNSS, which is expected to deliver enhanced performance capabilities over the next decade, given modernization plans for current constellations as well as emerging capabilities from 'new' constellations. Complementary technologies and techniques which can potentially, provide solutions that maintain or improve GNSS performance levels will also be discussed.

II. EXAMPLE
As an illustration of the possible transitions between environments a single scenario is presented based on the premise of an end-to-end journey, which will illustrate how people may migrate from one environment, and mix of positioning technologies, to another, with different conditions and technologies, many times during an entire journey. Consider people starting a journey, on business, from one location to another. They might start in an indoor, office, environment, or maybe from their home. In either of these cases they are typically in a well-known environment and would rarely need any PNT information to navigate and start their journey. However, an increasing number of travel planning applications will use the location of the travelers to start the process of selecting travel options. But, the accuracy requirements of these applications are very low, and are often simply satisfied by wireless access point locations, as GNSS would rarely be available. On leaving the building, at the start of the journey, it may be that the people transition to a mobile car, driving around a dense urban environment. It is quite typical these days that users will migrate their PNT capability with them through nomadic devices, such as smartphones. This recent evolution has seen fewer people using dedicated in-car navigation devices and more using their smartphones as a single device as they move between environments. The navigation applications running of smartphones have similarly evolved to emulate the characteristics previously found in dedicated in-car navigation systems, such as the use of map-matching and some additional sensor integration (i.e., using the accelerometers and gyros). Whilst in this phase of the journey it is possible that the PNT information is also being used to serve applications assisting the assessment of driving quality for insurance purposes, or for road tolling schemes. Some of these may be satisfied by the same nomadic device, while others may use additional dedicated equipment. All the PNT tasks will have differing levels of reliance on GNSS and have different requirements for accuracy and robustness.
Our travelers are heading for a rail station in order to catch a train for the next phase of their journey. Perhaps the next location based application they use would be to book and find a parking place adjacent to the rail station, which could be outdoors, in a multi-story building or even underground. It is very probable that the environment is, again, challenging for GNSS with high multipath and partial or complete obscuration. The nomadic device could now migrate to a different blend of measurements and sensors; in this case there could certainly be support provided by dedicated infrastructure. On leaving the car the travelers now enter a rail station, which is typically a large complex structure. The challenge for the travelers is to find their way around the station, to required services (maybe the ticket office, shops, and lavatories) and ultimately to the correct platform to catch their train. Rail stations are very harsh indoor environments with little potential for GNSS coverage and difficult signal propagation characteristics for dedicated or ad hoc radio positioning. But, the goal would be to use the same nomadic device to meet the changing PNT requirements. And finally, once on the train, the passengers may wish to firstly find their pre-booked seats and then later check on the progress of their journey. The environment and dynamics have once again changed radically. This time the immediate surrounding environment (the rail carriage) is moving with the passengers, and so conventional 'indoor' approaches are not relevant. A PNT application that is being considered by many rail companies is the concept of ticket-less travel. If it is known that a passenger was on a certain train, in a certain class, on a certain journey, then why is it necessary to issue a ticket for that journey?
The final stages of the journey may or may not replicate the initial stages. Perhaps the travelers took a taxi to their destination and this was booked using another location-based service. Throughout the journey a further PNT application could be the provision of the travelers' locations to crowdsourced travel information, and so benefitting other travelers in terms of congestion and future planning.

III. MULTI-CONSTELLATION GNSS HYBRIDIZATION
GNSS, of which the best known is the US Global Positioning System (GPS), essentially provide us with two pieces of information, position and time. GNSS-enabled devices now span almost all military and civil application domains, including: intelligent mobility, guidance, logistics, location based services, communications, commerce, precision agriculture and many areas of science and engineering. However, of growing concern is the vulnerability of the basic signals from GNSS satellites to interference, leading to disruption or potentially complete denial of PNT capability. The GNSS signals are inherently very low power and this leaves them open to both accidental interference (for example due to variations in atmospheric conditions and space weather events) and intentional interference through jamming and spoofing (for defense, criminal or simply personal privacy protection purposes). For many application sectors it is critical that robust PNT services are available that are resilient to interference, or other denial of service attacks, from whatever source these may arise.
Furthermore, multi-constellation GNSS is already a reality, as all new smartphones incorporate multi-constellation chips, and all top-of-the line geodetic/engineering/surveying receivers are multi-GNSS. If we acknowledge the vulnerabilities of a single constellation GNSS, then the question to ask is whether a multi-constellation GNSS approach can provide the levels of accuracy and robustness required for particular PNT applications. More particularly, does the hybridization of one (or more) GNSS to another provide sufficient redundancy, independence and spectral diversity to ensure continuous robust provision to meet service requirements?
It is an unfortunate fact that because of the limited bands of radio spectrum allocated to GNSS the sources of interference, which affect one system are very likely to affect another. The understandable desire to make different systems as interoperable as possible has also increased the vulnerability to common modes of failure across systems, due to interference and jamming. However, multi-constellation GNSS does increase the resilience, especially with regard to intentional spoofing of signals, as it would be a rather complex and sophisticated system which could simultaneously spoof multiple GNSS transmissions.
A key factor of the four operational and emergent satellite navigation systems is that they are completely independent in terms of their space and control segments. And it would be easy to assume that the use of multi-constellation GNSS would increase the overall redundancy in the event of a single constellation failure. The first real test of this hypothesis came on the 1 April 2014, when the entire GLObal NAvigation Satellite System (GLONASS) constellation was disrupted as illegal ephemerides were simultaneously uploaded to every satellite. The impact of the incorrect data continued for more than 10 hours, and although the satellite ephemerides were incorrect, the pseudo-ranges continued to be broadcast correctly. Blume et al. [1] reported on the impact on a number of different multi-constellation GNSS receivers, and evaluated the effects on the receivers' tracking and positioning performance. They observed that "for some receiver types the on-board receiver autonomous integrity monitoring (RAIM) failed to ignore the incorrect messages, resulting in degraded GLONASS and GPS tracking, and, in some cases, complete tracking failures and significant data loss. In addition, many of the receivers with clock steering enabled showed outliers in their receiver clock bias estimates that also coincided with the outage." Their investigations also showed varied responses across receivers from different manufacturers. It is evident that as manufacturers have striven to integrate systems as closely as possible within their receivers, this is potentially having a detrimental impact on the independence between the individual systems.
In civil aviation, RAIM [2,3,4] is an approved method for lateral navigation (LNAV) for the en route, terminal and nonprecision approach phases of flight and RAIM prediction is required if GPS is to be used as a sole means to satisfy area navigation (RNAV) requirements. The main drawbacks of this technique are that it uses a single constellation, is restricted to a single frequency and has the capability of detecting just single faults, which therefore severely limits the performance. Advanced RAIM (ARAIM) is currently under development to provide enhanced service provision, with the goal of meeting the requirements of LPV 200 (Localizer Performance with Vertical guidance, with the decision height of 200 feet above ground level) approaches. It is fundamentally based on multiconstellation GNSS, but also incorporates the use of dualfrequency signals and multiple fault detection. The GLONASS incident of April 2014 made it perfectly evident that the implementation of robust ARAIM techniques will be required in order to provide the robustness to complete or partial system failure.

IV. MULTI-SENSOR HYBRIDIZATION
As concluded in the previous section a multi-constellation GNSS approach to the provision of global PNT can potentially address some of the limitations and deficiencies of a single constellation GNSS, but there are still many circumstances, due to common-mode failures, such as interference, jamming and simple line-of-sight obscuration, when GNSS alone cannot meet the requirements for continuous, accurate and robust PNT services. It is in these situations, and especially in challenging environments for GNSS, when a multi-sensor approach to PNT can provide the required levels of robustness and resilience. The following sections will discuss in detail the sensors and measurements, and the possible levels of hybridization with GNSS. But, this is no simple panacea, with just a single compelling solution. There are possibilities of a complex mix of disparate technologies, across different environments, with different user dynamics and different levels of support from the surrounding infrastructure [5,6,7]. It will not be possible in this single, short review paper to adequately consider all possible dynamics, platforms and environments, and so a particular focus will be placed on just some of those which are most challenging to GNSS. This paper addresses the dense urban and indoor environments and the typical users and platforms, which operate in these environments, such as pedestrians and emergency first responders.

A. Infrastructure
The level of infrastructure provided in a particular environment can have a significant influence on the availability of potential navigation sensors to augment GNSS availability. It is convenient to break the level of infrastructure provided by different environments into three broad categories: with dedicated infrastructure, with ad hoc infrastructure or with no infrastructure at all.
There are certain environments where it is possible to provide dedicated facilities and equipment that would enable continuous positioning and navigation capabilities. This already occurs in some warehousing and other industrial manufacturing applications where the surrounding structures have been equipped with radio, acoustic or optical positioning systems. But, in these cases the user equipment is also dedicated to a particular task and this often limits the number of potential user terminals. It is possible to envisage how railway stations, airports, hospitals and shopping malls could also be equipped with dedicated infrastructure to support PNT. But, in these situations, the challenges shift to those of standardization of approaches and the required equipment that would need to be carried by the users, which would potentially include large numbers of the general public. In an ideal scenario services would be provided to typical smartphone platforms, and would operate across all such indoor environments.
Many dense urban and indoor environments do already have installed radio and other capabilities which could support PNT in an ad hoc manner [8]. Perhaps the most obvious example would be the widespread availability of WiFi signals. Clearly, these are installed primarily for communications purposes, but many services are now readily available which use these same signals to provide different levels of positioning capability, both indoors and outdoors, using just standard smartphones as the user terminals [9]. The lack of dedicated positioning architecture of the systems does, however, pose many challenges which must be addressed in order to provide the levels of accuracy and robustness that may be required. Also within this category of ad hoc infrastructure it is possible to include information about the environment such as road, street and building plans. This data can often provide valuable constraints on the viable positions of users, on their possible movements and to bound the growth of errors inherent in some dead reckoning sensors [10].
In some environments it cannot be assumed that there will be any support provided whatsoever from the surrounding facilities or structures, and these environments can prove to be the most challenging in which to provide robust PNT. For example, consider the scenario of a fire fighter entering a partially destroyed building, with all the buildings services already having failed. In this sort of situation it is necessary to consider only sensors that can be carried by the users or can be set up in an ad hoc manner. However, it is often possible to build on links between groups of users and develop networks of collaborative or cooperative positioning (CP). These networks can share information between users, share partial measurements from positioning systems, or through measurements of the distances between users build a web of relative position.

B. Platform Dynamics
In addition to the environment and available infrastructure, another important factor that will affect the hybridization approach is the anticipated platform dynamics, and fundamental to the platform dynamics is the concept of a reference trajectory. Any time there are intermittent sensor outages (which is almost certainly the case in situations that require a hybrid approach), it is helpful to have some means to determine a nominal or reference trajectory, which can be used to determine short-term motion in the absence of sensor measurements. From an estimation point of view, having a reference trajectory enables measurements to be relevant beyond the instant in time when they were taken. Often, integration algorithms combine navigation sensor measurements with a reference trajectory by using the measurements to estimate errors in the reference trajectory. Then, the reference trajectory, corrected for the estimated errors, gives the final trajectory estimate.
There are several types of reference trajectories that can be used. The first is to simply use a mathematical model, which is appropriate in situations where there is knowledge about the type of movement that a system can make. For example, a satellite in orbit can be modeled very well using orbital dynamics. If the system involves a vehicle driving on the road, one can make reasonable assumptions about the vehicle dynamics on a road and encode that in a mathematical vehicle motion model. A second type of reference trajectory involves the use of deadreckoning sensor measurements, such as wheel odometers for vehicles, step sensors for people, or air data computers (measuring velocity through the air) for aircraft, combined with the direction of motion measured by, for example, magnetometer, compass or gyroscope. These measurements all give information for how the particular system is moving, but do not provide any absolute positioning information. As such, they are useful for generating a reference trajectory. Note also that all of these sensors measure motion relative to some particular physical object (the ground for odometers and step sensors, and the air for an air data computer). A very common sensor used for obtaining a reference trajectory is an Inertial Navigation System (INS) that includes an Inertial Measurement Unit (IMU) (accelerometers and gyroscopes) and an inertial navigation algorithm. An IMU measures specific force and rotation, and this information can provide self-contained information about the trajectory. Inertial systems vary greatly in terms of dynamic range and sensor quality, and the platform dynamics and accuracy requirements will drive the type of IMU that can be used in any given application. One benefit of an IMU is that it is measuring inertial quantities, and it does not require contact with any external physical quantity (in contrast with the dead-reckoning sensor measurements described in the previous paragraph).
Expected platform dynamics will have a big impact on both the sensors used and the overall integration approach. Generally, highly dynamic systems (i.e., systems in which the dynamics are not very predictable) stress the ability to obtain a reference trajectory, so higher quality sensors (IMU) are required. Additionally, the type of integration level will be determined in some part by the platform dynamics.
When integrating an IMU with other sensors, there are generally three levels of integration. Loose integration refers to when the navigation sensors generate a position or a velocity, which is used to estimate the errors in the IMU. Next, tight integration refers to navigation sensors providing a raw measurement (rather than a position or velocity fix), which is combined with several other raw measurements to estimate IMU errors. Finally, ultra-tight integration refers to situations where the IMU data are used in the process of forming the navigation sensor measurements in the first place. If we consider the classic GNSS/IMU case, loose integration would combine position solutions from a GNSS receiver with an INSbased trajectory prediction, tight integration would combine pseudo-ranges and/or carrier phase measurements from a GNSS receiver with an INS-based trajectory, and ultra-tight integration would also use the INS-based trajectory to maintain a navigation solution which is used directly within the GNSS tracking loops internal to the GNSS receiver as it tracks the GNSS signals.

C. PNT Sensors and Techniques
To address GNSS reliability, there are two basic solutions: monitoring the received signal and using other sensory data. RAIM technology has been developed to check the consistency of the position solution based on pseudo-ranges [2]. Besides checking all the possible solutions, advanced RAIM implementations may detect system failure, such as a faulty satellite. RAIM is, clearly, a baseline technique and should be part of any more sophisticated solution. In fact, its importance will increase with the introduction of other GNSS systems, as the number of signals will grow, resulting in higher redundancy in position computation combinations. Note that the availability of multiple GNSS systems will also increase the resistance to system failures. By design, RAIM is dependent on signal reception, and provides no PNT alternative. In contrast, using sensory data from non GNSS sources can potentially provide both quality monitoring as well as PNT solutions for situations with poor signal reception, high interference, signal jamming, catastrophic system failure, etc. Sensor integration has been used in navigation for many years [11], and the classical GNSS/IMU integration based on the Extended Kalman Filter (EKF) is one of the most widely used methods for vehicle navigation. Sensor integration is a broad topic and, in this paper, it has been reviewed solely based on the sensors and methods. Table I lists all the relevant, both traditional navigation and imaging, sensors which are currently considered for PNT applications.
In Table I, X, Y, Z denote 3D Cartesian coordinates of the navigating platform; V X , V Y , V Z denote 3D velocity components in X, Y, Z directions; x, y, z are image coordinates of the features extracted from an image; ω, ϕ, κ denote heading, pitch and roll; α and β denote ray direction in the imaging frame, a 2D subset of the 3D attitude vector of heading, pitch and roll; a X , a Y , a Z denote 3D vector of accelerations measured by the accelerometers; ϑx, ϑy, ϑz denote 3D vector of angular rates measured by the gyroscopes; R denotes range; Z is altitude and N is the step count; CLOS stands for clear line of sight, CORS stands for continuously operating reference stations; GIS stands for geographic information systems; CAD is the computeraided design; RF is radio frequency; SLAM stands for simultaneous location and mapping; WLAN denotes wireless local area network, and QA/QC denotes quality control/quality assurance. Radio frequency (RF) sensors are fundamental to navigation, and systems are usually grouped into satellite based or terrestrial categories. In both cases, they rely on a network of deployed radio transmitter systems and supporting infrastructure. Most systems require CLOS (Clear Line Of Sight) operations, and the positioning solution is provided in a global or local navigation frame. IMU sensors are based on physical measurements and, in contrast to RF sensors, require no infrastructure. IMU technology has recently seen remarkable developments at both ends of the performance spectrum. Low cost micro-electro-mechanical systems (MEMS) IMU performance is improving, and approaching or even exceeding the tactical grade level (gyro accuracy on the order of 1/h). Cold atomic interference-based super high accuracy IMU technology offers gyro and accelerometer bias stability in the 60 μ/h in the near future [12]. In fact, a paradigm shift is happening, as the IMU is becoming the primary sensor for navigation, assisted by GNSS and other sensory data.
Another natural signal source that can be used for navigation is the earth's magnetic field, but not in terms of determining north (as has been done for centuries with magnetic compasses). Rather, it is through knowledge of the variations of the magnetic field, which change as a function of position. Most people have experimented with a compass indoors and have found that it does not always point north, depending on where you are standing in a particular room. This is due to variations in the magnetic field due to man-made structures, electrical currents, or other magnetic-field-perturbing effects. The concept behind magnetic field navigation is that if there is a map of the magnetic field variations in a particular area, then we can compare magnetic field measurements against a map to determine position information. This concept has been successfully demonstrated for indoor navigation [13], for navigation of vehicles driving on roads [14], and for aircraft [15,16].
Magnetic field sensors can be relatively inexpensive with small size, weight, and power requirements for use in smartphones. A common three-axis magnetic field sensor is a fluxgate magnetometer. This type of sensor is sufficient for most ground applications, since the magnetic field variations at ground level can be significant. However, for aircraft applications, more precise instruments, such as atomic scalar magnetometers, are required to detect the much smaller magnetic field variations present at flight altitudes.
Perhaps the biggest challenge to magnetic field navigation is the need to have a map of the magnetic field variations. For indoor applications, this can be obtained by walking through a building with a magnetometer. For vehicles driving on the road, the map can be generated by a vehicle when GNSS is available. Then, if GNSS is lost, the map can be used for positioning purposes. For aircraft navigation, an airborne magnetic field survey is generally required.
Another significant hurdle to overcome involves calibration of the magnetometer, particular in outdoor (road or aircraft) applications where the variations are not as strong as indoors. An important aspect of calibration is removal of the perturbing effects of the vehicle (or person) on which the magnetometer is mounted. There are a number of calibration procedures available, most of which entail rotating the magnetometer in an environment in which there are little or no magnetic field variations [17].
When compared to GNSS, performance of magnetic field navigation will vary significantly, depending on the variation in the magnetic field at any given location. This is particularly true of ground vehicle navigation. Testing of this approach with ground vehicles in a suburban environment demonstrated that very precise (approximately 1m accuracy) position fixes were possible, but that such fixes tended to be intermittent in terms of their availability [14].
Depending on the application, positioning using magnetic field variations may be a viable addition to a hybrid system, which will enable more robust overall navigation performance than a GNSS-only approach.
The use of imaging sensors in navigation is increasing since they are capable of providing rather accurate range and orientation data at a local scale. As sensing technologies have advanced and became widespread, the use of imagery has become ubiquitous, and imaging sensors are available in a broad variety of spatial, temporal and spectral resolutions. In addition, active sensors, such as LiDAR can directly provide range observations, including depth images. The current trend is to move toward crowd-sensing [18], which is primarily driven by the large number of smartphones, estimated to be 1.5+ billion as of writing this paper, and serves mainly for visualization and mapping purposes. The importance of this rapidly growing volume of imagery is that it is forming a continuously extending and improving object space description that can be used for navigation.

V. IMAGE BASED PNT
Image based navigation, using stars and landmarks, is as old as navigation itself, while modern image based navigation has a relatively short history. On the one hand, imaging sensor performance has been a key enabling technology allowing for information-rich data acquisition. On the other hand, algorithmic developments, in particular, advances in computer vision, have highly automated the extraction of geometrical information, making it feasible to efficiently integrate it into navigation filters [19,20,21,22,23,24]. Broadly speaking, imaging sensors work in active and passive modes, based on whether they provide a signal to observe the object space or just sense some part of the spectrum. Some of the active sensors, such as LiDAR, can directly provide 3D data that is advantageous as it integrates relatively well with conventional navigation systems [25,26,28]. The terms of image and terrain based navigation differentiate between using 2D image or 3D surface data, respectively. Using the most typical optical imagery, the 3D object space is projected to a 2D image plane, and stereo or multi-ray image processing is needed to recover 3D information. Note that 2D imagery can be used in mono mode too. Regardless of the image type, the primary processing of any imagery usually includes image feature extraction and matching that forms the basis for obtaining correspondence between image and 3D object space, and, consequently, deriving geometrical information for navigation. The general concept is shown in Fig. 1. In Fig.1, t i -t i+3 denote consecutive time epochs; δV and δϑ denote 3x1 vectors of velocity and attitude increments, respectively; T is the 3x1 translation vector with components t x , t y , t z ; R is the rotation matrix, with components r 1,1 through r 3,3 ; s denotes the scale; C is 3x1 vector of Cartesian coordinates of the matched features in frames i, i+1…used to calculate the components of matrices R and T (and s, if sufficient information is available); and are 3x1 vector of trajectory coordinates in frames i and i+1; f denotes the feature space with n i , n i+1 … elements at epochs/frames i, i+1…, respectively; δϑ and δt are the user's position and orientation change recovered from image matching; X, Y, Z denote Cartesian coordinates provided by GNSS, and Φ and P are GNSS carrier phase and pseudorange data, respectively. When considering a hybrid PNT system, the use of imagery offers an approach to obtaining additional non-GNSS measurements that can be very useful, depending on the application. Image measurements are available indoors (given sufficient lighting), where GNSS signals often are not. Additionally, image measurements are not affected by RF interference, so they often can form a good alternative approach to GNSS aiding.
There are three fundamental approaches to using images for navigation. The first, visual odometry is where subsequent images (usually taken by the same camera) are used to determine motion that occurred between the epochs when the images were taken. Classic visual odometry involves two images, but the concept can be extended to more than two images using bundle adjustment techniques [29]. Visual odometry approaches rely on matching features between images, which require some degree of overlap, i.e., the parts of the images that display the same scene. With no overlap, there is no ability to obtain a visual odometry solution. Because visual odometry approaches often use images from the same camera, taken at approximately the same time and from a similar angle (so lighting and viewpoint are similar), it is generally straightforward to match features between images. Visual odometry measurements can only determine how a platform/user has moved, but cannot determine the absolute position. As such, visual odometry can sometimes be used as a reference trajectory, or to aid the reference trajectory.
The second approach to image-aided navigation may be referred to as absolute positioning. With absolute positioning, one must have a database of image features and their locations. Then, when a new image is taken, the goal is to identify features that are in that image which match features in the database. Once this is accomplished, the camera's absolute position can be determined. Feature matching between the navigation camera and the database is usually more challenging in the absolute positioning approach, as compared to visual odometry, because the images used to generate the database were often taken under different lighting conditions, from different object distance or direction, or with different cameras. While feature matching is more difficult in absolute positioning scenarios, the benefit is that such an approach can provide an absolute position update, similar in nature to an update from a GNSS system.
The third approach to image aiding is a blending between visual odometry and absolute positioning, and it is often referred to as simultaneous localization and mapping (SLAM). With SLAM approaches, features are identified in a sequence of images, and both the feature locations ("mapping") and the camera pose (position and attitude-"localization") are simultaneously estimated. In the end, SLAM approaches enable absolute positioning if any feature (or set of features) has a known absolute coordinate. Otherwise, it tends to operate more as a dead-reckoning visual odometry approach, similarly to the bundle adjustment.
All of the above methods involve the identification of image features. These features are identifiable patterns within the image, and a common feature used in image navigation is the scale invariant feature transform (SIFT) [30]. SIFT features identify particular keypoints in the image and associate a 128 byte descriptor to each keypoint. SIFT features are designed to be scale invariant, rotation invariant, and invariant to modest amounts of affine transformation. The ability to have a descriptor for each keypoint enables robust feature matching, whereby feature matches are limited to keypoints that have similar descriptors. Note that often, other geometric constrains are also applied to keypoint matching. While SIFT features are common, there are other similar point features that can be generated, including Speeded Up Robust Features (SURF) [31] and Fast REtinA Keypoint (FREAK) features [32]. Additionally, some algorithms may use other forms of image features, such as lines or color segmentation.
One important characteristic of image measurements is that there is inherent scale un-observability when using a monocular camera. This is due to the fact that a given pixel in an image describes the quantity or color of light coming from a particular direction relative to the camera, but it does not, generally, give any information about the distance to that object. Imagine taking a set of images of a camera moving through a room. Next, imagine that this room could somehow be expanded by a factor of 10, so that everything in the room was exactly 10 times larger. Now, imagine that the same camera is moved through the enlarged room along a trajectory that is 10 times larger and 10 times faster than the original trajectory. In this case, the images between the original and the expanded room would look exactly the same, even though the actual scenes had different sizes. This demonstrates that the images themselves do not carry inherent scale information. In order to determine scale, additional information (with scale) must be added to the system in some manner. Examples include use of binocular cameras (with known baseline between cameras), stadiometry (performing object recognition of objects that have known scales), or identification of two features that are at known world coordinates (with a known baseline). One challenge with image measurements, as compared to GNSS measurements, is that image measurements are highly scene dependent. An aircraft attempting to navigate over an industrialized area has many potential identifiable features at known coordinates that can be used for absolute positioning, but the same aircraft flying over the ocean will have no features, which can be used for absolute positioning. It is possible to predict the performance of GNSS signals by knowing only which satellites are visible, but prediction of image navigation performance depends on knowing the nature of the exact scenes that will be observed, which is highly variable and difficult to model.
An example of indoor image-based navigation is shown in Fig. 2, which shows four different tests using a stereo vision navigation system, using two different IMUs (an inexpensive commercial grade IMU and a tactical grade IMU) [21]. In this test, a vision navigation system was pushed along a 180m path around a laboratory building. SIFT features were obtained from the images, and the stereo cameras were used to obtain scale. A SLAM approach was used, in which no prior knowledge of the environment was assumed. The system would track up to 10 simultaneous SIFT features, simultaneously estimating the locations of the features and the trajectory of the vehicle. Fig.  2 clearly indicates that the quality of the IMU was not a significant factor in the quality of the results, indicating that the vision SLAM measurements were the driving factor in overall accuracy. Overall, the system was able to maintain an accuracy of 2-3 m over the entire trajectory for each of the four tests. More details on this test can be found in [33].

VI. COLLABORATIVE NAVIGATION
Collaborative or cooperative positioning (CP) or navigation (CN) is a localization technique that emerged from the field of wireless sensor networks (WSN) [34]. Typically, the nodes in a WSN communicate with each other using wireless communications technology based on standards, such as Zigbee, WiFi, Bluetooth, 3G/GPRS [35]. This communications layer enables the sharing of data between the spatially distributed nodes in the network. In the context of positioning, the data being shared can be combined to estimate the positions of multiple nodes within the network or neighborhood. In this approach, the results are not only more robust when compared to independent solutions computed by individual nodes but more significantly, they afford enhanced positioning capabilities in difficult environments where for example, in the case of GNSS, there is multipath or complete obscuration of satellite signals. It has been demonstrated [36,37,38,39,40] that collaborative navigation not only enables navigation in environments where a single user cannot navigate, but also increases the accuracy of the navigation solution by several orders of magnitude. Similar to a loosely coupled architecture, the data shared can be simply positions and their variances as determined by the onboard sensors or, in a more tightly coupled architecture, the measurements made by each node can be integrated directly. Another advantage of the CP technique is that the communication RF signal itself can be used to derive internodal distances across the network. For example, in the case of intelligent transport systems (ITS), 5.9GHz dedicated short range communications (DSRC) can be used to determine the ranges between nodes in a vehicular ad hoc network (VANET) [41]. These ranges can then be used to further strengthen the positioning solution [42,43,44]. Some primary characteristics of the most commonly used techniques for ranging based on RF signals are listed here.
 RSS (Received Signal Strength): channel attenuation, which increases with distance, is computed from the known position of the transmitter and the received power  TOA (Time of Arrival): distance is computed by the signal's travel time as long as the network is synchronized  TDOA (Time Difference of Arrival): time difference of the TOA; eliminates the clock bias  AOA (Angle of Arrival): angle between the propagation direction of an incident wave and some reference direction. Figs. 3 and 4 illustrate a paradigm shift from single to multisensor, with an increasingly unconventional sensor configuration, to multi-platform CN. Fig. 4 depicts the concept of CN with the emphasis on transition between varying environments. In actual applications, the example networks are: soldiers, emergency crews, formation of robots or unmanned vehicles, etc., with the primary objective of achieving sustained level of sufficient navigation accuracy in GNSS-denied environments and assuring seamless transition among sensors, platforms and environments. The assumption is that at any given time, at least one of the layers in the system: ground-based, mid-air, GNSS, will have sufficient signals to guide the navigation of a mixed group of users, who supplement the solution by the inter-nodal range measurement.
The key components of a collaborative system, illustrated in Figs. 3 and 4, are (i) any GNSS signals and other sensory observations collected by each individual node that may be insufficient to generate an individual PNT solution, (ii) internodal ranging sub-system (each user can be considered a node in a dynamic network), (iii) optimization algorithm for dynamic network configuration, (iv) time synchronization, (v) optimum distributed GNSS aperture size for a given number of nodes (here, distributed aperture refers to the fact that, collectively, the users form a distributed aperture antenna), (vi) communication sub-system, and (vii) selection of master or anchor nodes. Sub-networks of users navigating jointly can be created in an ad hoc manner, as indicated by the ellipses in Fig.  4; notice that some nodes (users) may be parts of different subnetworks. In a larger network, the selection of a sub-network of nodes is an important issue, as in case of a large number of users in the entire network, computational and communication loads may not allow for the entire network to be treated as one entity. Still, information exchange among the sub-networks must be assured. Conceptually, the sub-networks can consist of nodes of equal hierarchy or may contain a master (anchor) node that will normally have better sensors and will collect measurements from all client nodes to estimate the CP solution.
As CP is typically based on the fusion of information at individual nodes and the inter-node ranges, the CP algorithms revolve around standard Bayesian estimation techniques such as extended and unscented Kalman filters or particle filters (see Tables II and III). The sensors used and their noise characteristics have primarily driven which algorithm is used. More recently, data driven filters as well as centralized and decentralized approaches are gaining prominence. These developments are an effort to address the challenges of network scalability and computational efficiency combined with an increasing demand for high performance positioning for safety and liability critical applications, [17,18,45,46,47,48].

VII. SENSOR INTEGRATION ALGORITHMS
There are a multitude of sensor integration techniques, and their use in navigation is generally specific to sensors, environment and motion patterns. In addition, the data integration can be done on several levels, such as raw sensory data, extracted features, identified objects, etc. Table II lists main integration solutions for IMU-based navigation using GNSS, UWB/PL (ultra-wide band/pseudolite) and image sequence generated fixes. As already indicated, the loose integration combines trajectory solutions while the tight integration combines sensor level data to obtain a trajectory, including position and attitude. In Table II

A. Primary Filtering Techniques Used in PNT
The trajectory estimation in navigation is generally based on filtering to create a solution using a dynamic model of the platform's motion in combination with sensory measurements. The characteristics of the motion play an important role in the selection of the filter, as in most cases it is based on a linearized model, such as the widely used EKF, which may not work well for high-dynamics motion in position and attitude. Table III lists major navigation filter solutions with their operational characteristics. Filters are increasingly combined with less conventional algorithms (e.g., knowledge-based) in real-time applications to increase robustness. For example, the Google autonomous vehicle uses conventional navigation filter for low level vehicle control while the overall navigation and situation awareness are based on a knowledge base that is continuously extended as feedback from experiences becomes available. The UKF is used to overcome the linearization approximation problems of the EKF, and is especially useful for handling higher order nonlinear systems. The UKF addresses the problem by using a stochastic linearization in contrast to the Taylor series expansion used in the EKF. It is arguable that the two main advantages of the UKF over the EKF are its accurate estimation and its easy implementation.

Particle filter (PF)
For highly nonlinear and non-Gaussian estimation problems, Sequential Monte Carlo Methods (Particle filter) can be used. The fundamental drawback of this approach is the fact that depending on the problem the PF analysis computationally expensive.

Non Bayesian techniques
Typically based on knowledge based systems using some learning mechanism, artificial intelligence techniques such as such as Neural Networks or Fuzzy Logic can be used to model complex platform dynamics such as in the case of personal navigation systems. In some applications these techniques have been combined with the EKF, UKF and PF demonstrating enhanced positioning and attitude estimation results.

B. Artificial Intelligence -An Alternative Integration Tool
The Artificial Intelligence (AI) discipline is primarily concerned with learning, knowledge, reasoning, and other processes of intellectual nature. The use of AI techniques in navigation goes back about a decade and was mainly prompted by emerging fields, such as autonomous vehicle navigation and personal navigation [49,50]. In general, AI may provide solutions for motion and environment monitoring processes that cannot be modeled by analytically, using traditional deterministic and stochastic models. AI techniques can be used directly to create a navigation solution, but they are more frequently used to identify the state of the platform motion. For example, correlating data from various sensors attached to a human body to the type of motion the person is performing can be useful to select the matching algorithm that provides the optimal model for that motion type. In a way, the human body can be considered as an additional sensor to support navigation. There are two essential approaches to acquiring knowledge: either from the prior experiences, that is through a learning process, or by formulating it based on the existing knowledge, provided by an expert. Artificial Neural Networks (ANN) methodology is based on a network formation that is built on learning. The number of layers and the number of nodes, etc., are optimized during a learning process where a statistically representative data set is provided with labels (reference information). Once the network is formed and tuned, it can be used, for example, to classify the motion type. In contrast, Fuzzy Logic (FL) provides a methodology to incorporate expert knowledge into a system that can do reasoning based on input data. A clear advantage of FL is that it can easily incorporate new knowledge, while ANN must be retrained if anything changes, such as new sensor data become available or the navigation environment changes. The personal navigator described in [51] incorporates both methods to model the pedestrian motion type and, in addition, to provide navigation solution indoors, where no fixes were available to control the IMU error growth. The most recent implementation of this system integrates image sensors, including 2D and 3D imagery to provide navigation solution [52]. More on pedestrian navigation can be found in [10,53,54].

VIII. EXAMPLE -REPRISE
So, let us now return to the opening scenario of travelers undertaking their journey across a variety of environments, through different modes of transport, and with ever changing PNT requirements. This paper has reviewed a range of possible technologies that can potentially augment GNSS to provide PNT solutions at the required levels of performance in these environments. Furthermore, it has been shown that it is not just the sensor technology that will change between environments, but the levels of infrastructure support, the dynamics and algorithms and software that are used to combine the sensor measurements together. This has led to the current situation of many different bespoke solutions, which have typically addressed just a single scenario, a single class of user and dynamics. The challenge is now to address the seamless transition, between environments, sensors and algorithms that allow our travelers the continuous end-to-end experience of PNT provision.
A typical current smartphone has many of the advanced positioning technologies described in the paper, and all enclosed in a small device. This includes multi-constellation GNSS, accelerometers, gyros, magnetometers, a barometer, one or more high resolution cameras and, of course, a variety of radio positioning capability, such as the cell phone itself, WiFi, Bluetooth and FM radio. And these are supported by a powerful microprocessor onboard the device. However, if this does not provide the required computation capability then, user, through the communications networks, could have access to the vast potential of cloud-based data and processing to augment the resident computational capacity. As a result, our travelers are probably already carrying most of the technology they require to enable continuous, seamless, robust positioning throughout their journey. And so, a question that, for now, remains to be answered is: will the next generation of smartphones offer a high accuracy multi-sensor fusion platform? If that is the case, then the main tasks will simply be to develop the PNT apps and assure cyber security to the users.

IX. CONCLUSION
This paper has taken a metaphorical 'step back' in order to provide a simple review of the technological advances that have taken place in PNT over the last two decades. It is patently clear that along with the now well-known, and quite dramatic, developments of GNSS, through both the maturing and nascent systems; there has also been a similar rapid, and parallel, progression of development of other PNT sensors. Advances in basic technology, including imaging, IMUs, magnetic and RF based systems, along with the quite astonishing additions of processing power have led to the evolution of practical and effective alternatives and augmentations to GNSS. But, the most important conclusion of this study is not simply that there are now viable alternative PNT technologies. It is only through the hybridization of a suite of different PNT solutions that we gain maximum benefit and can meet the challenging goals of continuous, accurate and robust service provision to the ever growing numbers of PNT users, which cannot be met by GNSS alone. A multi-sensor hybrid approach can address many of the vulnerabilities of GNSS, and provides us with a platform on which we can build future PNT capability. But, from a user point of view, this hybridization must be seamless, and provided through common, mass market, nomadic platforms. This will require system and software developers, and service providers, to face up to these significant challenges. Providing tailored or ad hoc solutions is no longer acceptable; the future of robust PNT will require a fully integrated approach.  in Electrical Engineering and Geo-Information Sciences from the Technical University of Budapest, Hungary. His research expertise covers broad areas of spatial information systems, LiDAR, high-resolution imaging, surface extraction, modeling, integrating and calibrating of multi-sensor systems, multisensor geospatial data acquisition systems, 2D/3D signal processing, and mobile mapping technologies. He has published over 300 peer-reviewed journal and proceedings papers, and is the recipient of numerous awards, including the 2009 APSRS Photogrammetric Award, several Lumley Research Awards from OSU, and various best papers awards.
He has been very active in ASPRS (American Society of Photogrammetry and Remote Sensing).