Crowdsourcing good landmarks for in-vehicle navigation systems

ABSTRACT Augmenting navigation systems with landmarks has been posited as a method of improving the effectiveness of the technology and enhancing drivers’ engagement with the environment. However, good navigational landmarks are both laborious to collect and difficult to define. This research aimed to devise a game concept which could be played by passengers in cars, and would collect useful landmark data as a by-product. The paper describes how a virtual graffiti tagging game concept was created and tested during on-road trials with 38 participants. The data collected in the road trials were then validated using a survey, in which 100 respondents assessed the quality of the landmarks collected and their potential for reuse in navigation applications. Players of the game displayed a consensus in choosing where to place their graffiti tags, with over 30% of players selecting the same object to tag in 10 of the 12 locations. Furthermore, significant correlation was found between how highly landmarks were rated in the survey and how frequently they were tagged during the game. The research provides evidence that using crowdsourcing games to collect landmarks does not require large numbers of people, or extensive coverage of an area, to produce suitable candidate landmarks for navigation.


Introduction
GPS-based in-vehicle navigation systems (IVNSs) are popular and widely used in automobiles today, typically existing as factory-fitted units, mobile (nomadic) devices and smartphone applications. Mobile human-computer interaction (MHCI) research in this area has primarily focused on issues of usability and efficiency, and the effects on navigational effectiveness and driving performance (Green et al. 1995). Concerns have also been raised regarding the deleterious effects of prolonged use of IVNSs, such as geospatial uncertainty and environmental disengagement (Leshed et al. 2008;Smiley 2000). It has been suggested that utilising landmarks to act as navigational cues within IVNSs may ameliorate these problems in addition to enhancing the usability and efficiency of these devices (May, Ross, and Bayer 2004).

Landmarks
Landmarks (defined here as 'an object in the landscape, which, by its conspicuousness, serves as a guide in the direction of one's course', OED 2015) are point-references that are external to the observer (Lynch 1960). They contribute to the acquisition of spatial knowledge about an environment (Darken and Sibert 1996), and are frequently cited during the ad hoc provision of directions (Burnett, Smith, and May 2001). Combined with directional instructions such as 'turn right', landmarks can ease route-following by indicating when and where these actions should be taken.
There has therefore been significant research interest regarding the inclusion of landmarks as navigational cues within IVNSs. This research has consistently demonstrated that augmenting navigation systems with landmarks improves their usability and utility, and aids route-following. For example, it has been shown that using landmarks in IVNSs is likely to improve navigation and driving performance, increase driver confidence, improve engagement with the environment, promote enhanced spatial learning and reduce reliance on the system (Oliver and Burnett 2008).
Despite their omnipresence in the natural driving environment, there are, however, significant obstacles associated with utilising landmarks as navigational cues within navigation systems; for example, identifying the most appropriate landmark to use at navigational decision points, and the laborious nature of collecting, describing and maintaining a database of these. Attempts have been made to extract candidate landmarks from existing datasets, such as points of interest or webbased pictorial databases (Elias 2003;Raubal and Winter 2002), but this fails to consider issues of relevance to the navigation task and selecting appropriate nomenclature.
Moreover, although some features that make a landmark good for navigation are quantifiable (e.g. advance visibility, spatial extent and permanence), many others have a subjective element and can only be 'quantified' in a specific context (Burnett, Smith, and May 2001;May, Ross, and Bayer 2004), leading to Raubal and Winter (2002) concluding that the suitability of a good landmark is 'a relative property'. Subjective elements include semantic salience (the importance assigned to a landmark as a consequence of the meaning it has to an individual), uniqueness and usefulness of location.
It is, therefore, suggested that a more effective approach is to collect navigation-based landmark data empirically using mobile crowdsourcing (Matyas et al. 2008;Winter et al. 2011).

Crowdsourcing
Crowdsourcing is the collaborative process by which groups of people contribute to the creation of shared content or datasets, particularly on a large scale. It is based on the premise that although individual contributions may vary in quality and content, the overall corpus will be a match for expert judgement (Howe 2008). The process harnesses the power of large numbers of people to perform tasks which computers find difficult, leading to it being colloquially referred to as 'Human Computation' (von Ahn and Dabbish 2004). There are many examples of using web-based crowdsourcing. A current, popular example is Zooniverse (2015), which requires the active participation of human volunteers to complete research tasks involving large datasets from diverse disciplines, including astronomy, ecology, cell biology, humanities and climate science.
Crowdsourcing is also particularly suited to the collection of geographical data on a very large scale. For example, OpenStreetMap (2015) aims to provide a free wiki map of the world. Given the likely cultural and geographic diversity of contributors, crowdsourcing also enables the collection of cultural and social/human aspects, thereby giving meaning to contributions (Winter et al. 2011). Moreover, given that the world is now suffused with web-enabled and sensor-rich mobile devices, there are plentiful opportunities to extend crowdsourcing to users of mobile devices as they go about their work-a-day lives. A good example of this is Waze (2015), a mobile crowdsourcing navigation application that allows drivers to feedback traffic/map problems discovered while driving, thereby providing upto-the-minute navigational advice to road users based on live road conditions. A critical aspect in the success of crowdsourcing applications is in their ability to motivate users to continue to provide data. Waze (2015) attempts to achieve this by rewarding contributors with credits when they travel over particular sections of road or report traffic problems. An alternative method to encourage/motivate participation is to provide game-like rewards or experiences to contributors. This approach, pioneered by Luis von Ahn with 'Games with a Purpose' (von Ahn and Dabbish 2004), attempts to solve largescale computational problems collectively through online gaming. A notable example is the ESP Game (von Ahn and Dabbish 2004) that randomly pairs players together to label images. Similar examples allow participants to cooperatively sort music clips into genres, or provide definitions for words.
Previous attempts at using a mobile game to collect geographical data, such as landmarks, have received limited success (Matyas et al. 2008). This has been attributed not only to the lack of clarity regarding how the data might be repurposed, but also to difficulty in developing a strong, high-level game concept, which would motivate the collection of good quality landmark data: attempts have also tended to concentrate on pedestrian navigation applications. In contrast, we develop and utilise a crowdsourcing game to capture good quality landmark data to enhance in-vehicle route-finding.

The study
This paper describes the development of a games concept, designed to be deployed in a vehicle context, and played by passengers, to collect and record ('tag') landmarks during routine journeys. A road study was conducted, in which the front-seat passenger played the game while being driven within an urban environment. The landmarks highlighted by participants during the study were extracted and validated by comparing them with independent quality ratings made by respondents to an online survey.

Graffiti tagging game
Building on approaches commonly employed in both game design and MHCI, a games concept prototype was designed and developed using a user-centred, iterative, prototyping and evaluation process. The aim of the graffiti tagging game was to allow passengers in vehicles to place a graffiti tag on objects along the route, which they believed would act as useful navigational cues in an automotive situation, using a simple interaction style, that is, by taking a picture and then entering a description, or tag. The prototype was created as an Apple iPhone application (app) using Xcode on an Apple Mac computer.
During the game, players were presented with the iPhone camera view screen, overlaid with a pre-drawn graffiti tag. Players were able to use this to view their surroundings and preview how the tag would appear on different objects ('landmarks'). To place a graffiti tag, players were required to position the phone so that the generic descriptor was located over their chosen landmark and then press the 'Tag Here' button on the screen. In so doing, the phone recorded a geo-located image and presented the player with a text-entry screen where they were required to enter a description of the object they had tagged. Players were encouraged to limit descriptions to two words, although this was not enforced, thereby allowing players the freedom to offer additional (cultural) meaning to locations and landmarks, if so desired. After players had entered the description, they selected the 'Done' button and the app returned the player to the main 'home' screen, ready to tag an object at the next location (see Figure 1).

Participants in-road trial
Thirty-eight people took part in the on-road graffiti tagging game (22 males, 16 females), selected primarily from the staff and student population at the University of Nottingham. The mean age of participants was 24.7 years; ages ranged from 18 to 56 (SD = 7.4) years. Eight participants were non-drivers and 30 had some driving experience. Forty-seven per cent of the participants indicated that they played mobile games at least once a week. Participants received a £10 (GBP) shopping voucher as reimbursement for their time.
Route followed during road trial Players sat in the front passenger seat of the vehicle (front-left) and were driven along the route by the experimenter. An urban route was chosen, approximately four miles in distance, which took about 20 minutes to complete under normal traffic conditions. The route began at the University of Nottingham Jubilee Campus, continued to the centre of Nottingham and then returned to the Campus via an alternative route, thereby ensuring that no road was traversed more than once. The journey included 12 navigation decision points and comprised a number of different types of roads and junctions. Participants were required to leave graffiti tags at each of the 12 navigation decision points along this route. These comprised 'choice points' (where drivers were required to select a new route/road), 'potential choice points' (where new routes/roads existed, but drivers were not required to follow them) and 'on route points' (where no new routes/roads existed, but it was felt appropriate to confirm to drivers that they were on the correct route) (Lovelace, Hegarty, and Montello 1999). On entering a designated graffiti zone, participants were alerted by the sound of a buzzer (activated by the experimenter), indicating that they should select, tag and capture an appropriate landmark. After the journey, players completed a brief questionnaire to provide feedback on their experience, such as their reasons for choosing each of the landmarks.

Landmark evaluation
A questionnaire survey was conducted to identify which of the landmarks chosen during the graffiti tagging game would be the most suitable as navigational aids, and to determine if those landmarks that were most frequently tagged were likely to be the best aids. Images of the landmarks selected during the graffiti tagging game were presented on a web-based survey in the same order that they appeared along the route. For each navigation choice point, the top five objects receiving the highest number of tags during the game were presented. Google Maps Street Views were added to the questionnaire to provide context for each landmark (see Figure 2). This precluded several landmarks, where the view had altered since the Google Maps images had been captured. Checks were also made to ensure that the chosen landmarks were equally visible from both the Google Street View image (captured from an externally roof-mounted camera) and by drivers/passengers during the game (i.e. from within the car).
Survey respondents were asked to rate the suitability of landmarks using five factors compiled from relevant literature (Burnett 1998;Burnett, Smith, and May 2001;Duckham, Winter, and Robinson 2010): visibility (encompassing size, prominence, advance visibility and night-time versus daytime visibility), usefulness of location, uniqueness, semantic salience (familiarity, ease of naming) and permanence. Explanatory notes were provided for each factor to aid understanding. Respondents were also asked to provide an overall rating. For all questions, 5-point Likert-style scales were used, where 1 was 'never suitable' and 5, 'ideal', based on the scoring system used by Duckham, Winter, and Robinson (2010).

Respondents to questionnaire
A total of 100 people (44 males, 56 females) responded to the survey, which was advertised online using the University intranet and took approximately 15-30 minutes to complete. Respondents mainly comprised staff and students at the University of Nottingham. The mean age of participants was 26 years; ages ranged from 18 to 63 (SD = 10.0) years. Sixty-eight of the respondents were drivers and the remainder had no driving experience. As an incentive, respondents were entered into a prize draw and a donation was made to a charity, selected by respondents, for each completed survey.

Results and analysis
Graffiti tagging game During the road study, 452 objects were selected and tagged. Results are presented using the following themes, which emerged during analysis: consensus, type of object chosen, signage, position, permanence and association with journey. Players were also asked to explain their motivation for selecting different objects. Further commentary is provided regarding individual differences (gender, driving experience and route familiarity) and how this influenced the objects selected. The results and analysis of the graffiti tagging game conclude with a brief discussion on tag descriptors and general comments made by participants.

Consensus
There was a high degree of consensus between different players. In 10 of the 12 locations, over 30% of players chose the same object to tag; in 3 of these, 50% or more chose the same object. Popular objects were the Nottingham Evening Post Offices, the Britannia Hotel and Raleigh Park apartments. The number of different landmarks chosen at each location varied, ranging from 6 (at location 10) to 16 (at location 1); the mean was 10.8 (SD = 2.9).

Type of object chosen
In every location, the most popular type of object chosen was either a building or a structure. Buildings were also the second most popular choice in all but one location. Players reported that larger buildings presented the best surfaces on which to place tags ('Basically it was the nearest large building -I could tag the whole building with one tag', 'Big walls are more tempting'). In some situations, street furniture, walls, fences and billboards were chosen, but these were much less common. Traffic lights were the most popular of these, but even so were only seldom chosen.

Presence
The presence of large distinctive name signs, such as those found on the Evening Post Offices and the Britannia Hotel, also seemed to play an important part in encouraging participants to tag buildings -9 of the 12 top choices had prominent signs displayed on them. Such signs also provided an easy and accessible description for players to use.

Position
The position of objects appeared to influence choice. There were three key aspects: position in relation to car, position in relation to road and position in relation to decision point/trigger. The most popular objects selected by players were located in front of the vehicle (9 out of 12 locations) ('Richmond House stood out as being straight ahead') and in close proximity to the roadside (10 out of 12). For the remaining zones, the most popular choices were located to the nearside of the vehicle (all players were travelling in the front passenger seat of a right-hand-drive car), though players commented that, on occasion, these were difficult to capture ('I tried to tag something through the side window but I'm not sure I got it'). None of the most popular choices were located to the offside of the vehicle. Players rarely chose distant objects, with notable exception of large or significant landmarks, such as Nottingham Castle. In making selections, most of the players interpreted the buzzer as a signal to immediately find an object to tag. On a few occasions, however, players took longer to select a target. Consequently, on seven occasions, the object chosen would not have been visible at the point that the buzzer was originally sounded.

Permanence
Despite clear instructions to tag permanent features -'the sort of things that will be there when the next player comes along'a significant minority of players (28 of the 452 tags placed) chose transient items, such as vehicles and pedestrians and even clouds/sky, to locate tags.

Association with journey
Forty-eight or 10.6% of the objects that were tagged could be described as being associated with the journey experience, for example, traffic lights, road signs or petrol stations. The most popular of these was 'traffic lights', which was tagged 17 times. 'Traffic light' was also the most used phrase in the descriptions, appearing 12 times.

Motivation
The most popular reason given by participants for selecting specific objects were size, colour and contrast, familiarity, novelty and fantasy.
Size: Size was frequently mentioned both as a factor in making an object distinctive from others and also in making it easier to tag with virtual graffiti. In the graffiti tagging application, the tags were fixed in size, and the players also had to contend with the movement of the vehicle. This may have led to the predominance of walls and larger buildings over roadside furniture and road signs in the choice of objects tagged ('I wanted something that was big and that I could fit the whole tag on', 'it was the largest static objectneeds to be quite large because of the size of the tag').
Colour and contrast: Colour and contrast were common factors influencing choice. A tyre depot and carpet warehouse (tagged 14 and 18 times, respectively) were both popular choices, due to their bright orange facades ('compared to the other buildings that one is really bright'). Some players also mentioned looking for backgrounds that would provide a good contrast for their tag ('on the yellow part, cos the tag is red', 'nice white space to stick the tag sign'), suggesting that tag colour (which was red in the game) and size may also influence the choice of objects to tag in virtual graffiti tagging games.
Familiarity: Landmark choice was also influenced by familiarity. For example, buildings that held particular meanings/associations, including university buildings, snooker halls and the police station, were mentioned ('my personal tutor's in that building', 'I've played snooker there'). The popularity of Raleigh Park (student accommodation), which was tagged by 20 of the 38 players, may have been influenced by the large student contingent that took part in the study.
Novelty and fantasy: Another factor that influenced tagging location was the notion of placing tags on novelty or high status places such as the Police Station, the Magistrate's Court or pubs ('I thought it would be cool to tag a pub', 'If you know the city … there's a genuine desire to put my graffiti on particular buildings. For example I have to put graffiti on the magistrate's court'). Some players also stated that they intentionally attempted to select unusual objects or vary locations in order to add additional challenges to the game ('I was just trying to be a bit different'), or choose objects that they thought a real graffiti artist might adorn, such as street furniture, blank walls and billboards. This suggests that the game supported the fantasy of being a graffiti artist ('It looked run downit looked like it should have graffiti').
Gender, driving experience, route familiarity Finally, consideration was given to the individual differences of participants to investigate whether this influenced the landmarks chosen. Paired t-tests were conducted to compare players' gender, driving experience and route familiarity. A significant difference was found regarding the most popular objects tagged by male and female participants (t(36) = 3.615, p < .05), with male players selecting the most popular choice more often in all 12 locations. In contrast, female participants were more creative when selecting landmarks.
There were no significant differences between drivers and non-drivers, though it is recognised that there was a relatively small number of non-drivers who took part. Players who were unfamiliar with the route showed an increased tendency to choose landmarks with highly visible signs, but these types of buildings were also highly popular with players in general.

General comments
During the post-study interview/questionnaire, the majority of players indicated that the virtual graffiti game was engaging and enjoyable ('It was good fun'; 'Takes you back to being a kid'). When asked to rate the experience on a 5-point scale of enjoyment (from 1 = 'no fun' to 5 = 'great fun'), 87% of players gave the experience a rating of 3 or more, indicating that the experience was 'fun'. Other benefits were also apparent: 'It really gets you noticing the landmarks'.
Players were also asked what might encourage them to play a virtual graffiti game. Responses were invited based on the motivations of real-life graffiti artists (creativity/self-expression, social play, leaving your mark, combating boredom, competition and legal rebelliousness) (Halsey and Young 2006). The highest rating was given to creativity/self-expression (mean ranking 2.3 out of 6, where 1 is high) -'becomes like an art gallery almost'. The potential of social play was also rated highly (mean ranking, 2.9) -'you're building up a kind of online following of people who tag regularly', although this functionality was notably not implemented in the prototype.

Landmark evaluation
Mean scores for each of the five individual factors (visibility, usefulness of location, uniqueness, semantic salience and permanence) and the overall rating were calculated for each of the landmarks used in the survey. The mean overall score (from the survey) was then compared with the number of players tagging an object during the road study to investigate the correlation between survey responses and road study tagging frequency.
The highest scoring landmarks were large, non-domestic buildings or structures, often with prominent signs, such as retail establishments, commercial buildings, public houses ('pubs') and sculptures (see Table 1).
The objects that received the highest overall ratings were also rated highest for uniqueness, suggesting that this was an important factor when selecting landmarks also noted by Burnett (1998) and Burnett, Smith, and May (2001). The size and colour of the landmarks appeared to be important factors in influencing visibility and distinctiveness. In several locations, the most popular objects chosen were physically much larger than other nearby objects (locations 1, 4, 5, 6, 7 and 12) and/or more visible or distinctive, for example, brightly coloured (locations 4, 5, 10 and 11). In several locations (e.g. 4 and 6), the most popular objects were located on the opposite side of the road to the turning, suggesting that either there was no other suitable landmark nearby, or that there were other factors influencing the choice.
Of particular interest is that, for 8 of the 12 locations, the highest rated landmarks corresponded with the most popular objects that were graffiti-tagged in the road study; in 2 of the remaining 4 locations, the highest ratings were given to the second most frequently tagged object. The relationship between the mean overall ratings obtained during the survey, and the number of players tagging those landmarks with virtual graffiti in the road study, was therefore investigated using Pearson's product-moment correlation coefficient, which revealed a strong positive correlation between the two variables (r = .624, n = 56, p < .01). There were also significant correlations between usefulness of location (.608), visibility (.585), uniqueness (.587), ease of naming (.572), permanence (.469) and the tagging frequency (all n = 56, p < .01). There were also very high correlations between each of these factors and the overall ratings (ranging from r = .823 to .943(56), p < .01), suggesting a possible halo effect, where respondents developed an overall good impression of the landmark and rated accordingly for all scales.
To investigate the correlation between survey responses and road study tagging frequency, the mean overall score (from the survey) was plotted against the number of players tagging an object during the road study (see Figure 3). It is evident that some landmarks that were rated highly in the survey were only tagged a few times during the graffiti tagging game (e.g. White Horse Pub, International House and Sir John Borlase Warren Pub). However, all of the landmarks that were tagged frequently (>30%) were also rated highly (>3.0) (e.g. Raleigh Park, Evening Post Offices, Britannia Hotel and National Tyres). Nevertheless, some of these landmarks failed to score proportionately better during the survey than other landmarks that were less frequently tagged. The absence of any data in the upper left quadrant of the graph in Figure 3 shows that there were no objects that scored poorly in the survey, but achieved more than 30% of the graffiti tags. The low level of tagging for some of the highly rated landmarks (e.g. International House) is likely to be due to the unusually high number of other, suitable candidate landmarks at these locations. For example, at location 1, only one landmark scored less than three for overall score in the survey.
It is evident from the scatter plot in Figure 3 that an exponential trend line provides the best fit (R 2 = .4842). Thus, we propose the following relationship between the percentage of players tagging an object (during the graffiti tagging game) and the overall rating of a landmark (as determined by questionnaire respondents) where x = mean overall landmark score. To define what constitutes a good overall score, we considered the distribution of the frequency of the scores recorded during the landmark evaluation survey. Consequently, we define a 'good' score as one that falls within a standard deviation of +1, equivalent to a value of 3.56, thus % TAG ′ D = 0.0082 × e 0.8885 ×3.56 = 19.4. ( This suggests that an object that was tagged by 19.4% (or more) of players at a certain location during the graffiti tagging game would be rated highly (≥3.56) and thus is likely to be a good candidate as a navigation landmark at that location.

Discussion
The graffiti tagging game was successful in achieving a consensus regarding the best objects to tag with virtual graffiti at navigational decision points ('it was the only obvious place'). This was influenced by the size, contrast and visibility (e.g. colour) of the objects, leading players to tag objects which would also be likely to fulfil the requirements for good navigation landmarks. Roadside furniture associated with the journey, such as traffic lights or road signs, was seldom chosen, although the frequency of tagging these objects increased when larger buildings or other obvious candidates were not available. It is suggested that larger buildings and objects were chosen in preference to roadside furniture because they were easier to spot and tag from a moving vehicle, and many also had distinctive signs that enabled them to be easily identified and described. In contrast, roadside furniture was not necessarily of a shape/size that provided a suitable canvas on which to tag, particularly from within the confines of a moving vehicle. It is noteworthy that the mobile application used for the study did not allow the graffiti tag to be altered or resized during the tagging process, and this may have encouraged the selection of larger objects. Thus, it is possible that some objects were chosen based on their physical dimensions rather than their suitability as navigational landmarks. This needs to be considered during further analysis/work. The most popular objects appeared to be those present in the forward facing view. However, some players commented that in a real-life situation they would be more inclined to look around for interesting objects. The graffiti game also appeared to encourage the selection of objects close to the roadside. Usefulness of location is a key requirement of navigation landmarks (Burnett 1998;May, Ross, and Bayer 2004) and this, therefore, has implications in determining landmarks that are most suited to navigation decision points.
It appears that the type of player (gender, driving experience and route familiarity) had some bearing on the objects chosen, but this was only found to be significant for gender, with females choosing a greater variety of objects. Players who were unfamiliar with the route showed an increased tendency to choose buildings with highly visible signs, but these types of landmarks were also highly popular with players in general. It was also found that there were no significant difference between drivers and non-drivers. If this was indicative of a wider population, it could be an important finding, as it would provide a larger cohort (including non-drivers) to contribute in the collection of navigational landmark for drivers.
The two-word descriptions provided by the players offer a good means to present the landmark within a real IVNS (for example, incorporated within verbal instructions), especially when combined with an image of the specified landmark. Furthermore, the descriptions captured during the study contained details of the type/ class of landmark as well as specific names/labels, and this supports a broader strategy for the selection of suitable landmarks for navigation purposes. During the game, the names of buildings with prominent signs appeared frequently, but the most popular description provided by players was 'traffic light'. This was despite the fact that there was only one location where traffic lights were tagged by more than three players. It is suggested that this is due to the ubiquity of traffic lights on the selected route, and throughout the urban road environment in the UK more generally. Consequently, 'traffic light' appeared frequently as a lower ranked choice in many of the locations, but was seldom ranked as the most popular choice. Traffic lights are often used during the routine provisions of directions (Burnett, Smith, and May 2001), so their appearance in the current dataset is unsurprising.
Commonly, the most popular choices had several of the important attributes of effective landmarks already highlighted within the literature, for example, permanence, visibility, ease of description and uniqueness. The relatively small number of roadside objects (or landmarks 'associated with the journey') chosen may be a problem. Such items are widely used when providing directions (Burnett 1998), and it is therefore suggested that they may be more frequently chosen on a less urbanised route where larger landmarks are not present. Further work should consider other settings.
The types of landmark rated highly in the questionnaire survey matched well with those that were most frequently graffiti-tagged, although some contradictions were evident. For example, public houses scored well in the survey, but were not popular places to tag during the road study. This may be due to the concentration of other suitable candidates in those locations. The strongest correlations were found between 'usefulness of location' and tagging frequency, although all factors were rated very similarly, suggesting a possible halo effect. As such, it was felt that no separate factor analysis was justified.
The data suggest that an exponential relationship provides the best fitting model for comparing tagging and landmark quality, revealing that an upper limit exists for mean overall landmark score which falls short of 'ideal'. This suggests that no landmark could ever be considered to be ideal or perhaps respondents were wary of using the extremes of the rating scales when scoring landmarks, suggesting a 'central tendency bias' (Couch and Keniston 1960). In any case, the proposed model should be treated with caution, as the graffiti trial data were not collected under controlled conditions. Furthermore, the regression model should not be extended beyond the scope of the data collected and landmarks rarely received very high or very low ratings. Nevertheless, it is suggested that the regression analysis presented is useful in giving an indication of the number of people required in order to collect useful landmark candidates. The model obtained indicates that any object which was tagged by more than 19% of players is likely to be a good candidate as a navigational landmark (i.e. would score 3.56 or greater out of 5). This tagging percentage figure is fairly low, indicating that the virtual graffiti concept may represent a highly effective means to extract suitable landmark candidates.

Gameplay
It was evident that the graffiti tagging game prototype was well received and provided valuable insights into the capturing and classification of roadside landmarks for navigational purposes. It employed a simple 'pointand-shoot' interaction style, provided a fantasy element of 'being a graffiti artist', which appealed to many of those who took part, and was described as enjoyable to play. A possible criticism is that the activity failed to achieve the status of a 'game' as it had no competitive element and provided no score or feedback on performance. Future versions could address this by adding a competitive or social aspect. For example, at a given navigational decision point, players could be challenged to guess what the current most popular tag was, or a community of 'taggers' could be created. Additionally, incentives could be provided for repeated/continued play, such as rewarding contributors or successful predictions with credits.
A further criticism may be that by forcing players to use the app in specific locations (i.e. following the buzzer activated by the experimenter), ecological validity was compromised. However, this was a necessary part of the experimental design in order to ensure that data were captured and could be compared at consistent locations. In a real-world situation, one would expect players to exercise free will when choosing tagging locations.
Finally, due to the highly distracting nature of the game, it is naturally intended to be played by car passengers rather than the driver. This may create a bias towards the selection of landmarks that are most visible from the passenger's seating position, rather than from the driver's, and one may therefore question the value and utility of the chosen objects as drivers' navigational aids. However, the evidence suggests that the most popular tagged objects were, for example, large buildings that are likely to be equally visible from both the driver's and passenger's viewpoint. It is therefore suggested that this concern is more theoretical than real, and is not expected to cause any problems in the practical application and presentation of the collected landmark data during driver-centric navigation advice.

Conclusion
In this paper, we have presented a virtual graffiti tagging game, described as engaging and enjoyable by players, that was designed to collect and tag landmark data, and was subsequently tested during an on-road study with 38 passengers. The data collected during the study were then validated by means of an online questionnaire survey, in which 100 respondents assessed the quality of the landmarks collected and their potential for reuse in navigation applications. The results have positive implications for the graffiti tagging game as a means of crowdsourcing landmark data. In particular, the frequency with which an object is tagged appears to be a strong predictor of good navigational landmark candidates for IVNSs.
Future work should seek to enhance the game concept, for example by adding a competitive element, as well as providing further validation of the collected data. This could be achieved by collecting data in other environments and also integrating the accumulated landmark data within a real-world navigation application.

Disclosure statement
No potential conflict of interest was reported by the authors.