1. INTRODUCTION
Navigation services have become one of the most widely used types of Location-Based Services (LBS). Many of the currently available navigation systems provide users with turn-by-turn navigational instruction, in which directions are associated with distances, e.g. “turn right after 250 metres”. Such a navigational strategy works very well for machine/robot navigation, as sensors sense and measure distances and headings. However, people give navigational instructions in a slightly different format. People associate directions with visual cues, such as “turn left at the church”. Such visual cues utilise easy-to-recognise and unique features and objects.
This strategy might be more attuned to the interests of pedestrians, since they move with relatively lower speed; therefore they can notice visual landmarks easily. In addition, such unique and easy-to-recognise features help people to memorise the path they have taken and also give a better understanding of their surroundings, especially when they are exploring unfamiliar environments. Landmarks are also interesting features in their own right for many people, such as tourists.
Landmarks are available both inside and outside buildings. Therefore any positioning technique that localises users with respect to landmarks, whose locations are usually fixed, can potentially provide seamless (indoors and outdoors) positioning solutions. Seamless indoor and outdoor positioning is one of the most challenging parts of pedestrian navigation. Seamless navigation is the topic of many research projects (Cheng et al., Reference Cheng, Yang, Li and Zhang2014; Zhang et al., Reference Zhang and Xu2012; Li et al., Reference Li, Wang and Li2013; Hansen et al., Reference Hansen, Wind, Jensen and Thomsen2009). In this regard, landmarks can be used as reference points in positioning techniques, which may then provide a seamless positioning solution. This paper proposes and implements one technique, which uses photographs of landmarks to localise users.
In contrast with drivers, pedestrians have a higher degree of freedom in their movements. They can walk across open areas such as squares, parks, grasslands or pedestrian malls, which can be traversed freely in any direction (Gaisbauer and Frank, Reference Gaisbauer and Frank2008). As current turn-by-turn navigational instructions designed to be given to vehicle drivers are mostly based on graph-based or street network-based algorithms, this way of navigating is not fully suitable for pedestrians as they do not only move on streets (Pielot and Boll, Reference Pielot and Boll2010; Amirian et al., Reference Amirian, Basiri, Gales, Winstanley and McDonald2015). Therefore landmarks can be used to provide seamless positioning and more desirable navigation services to pedestrians, who, unlike drivers, can go into and through buildings to get to their destinations, who move at a relatively low speed and for whom the purpose of their travel might not only be to get to the destination, but also to explore an unfamiliar area and indulge in sightseeing along the way. Landmarks could be assets in memorising an unfamiliar environment where users visit for the first time. Also providing information on landmarks (such as name, type, colour) makes the user sure that he/she is taking the correct path and has not got lost, as the user sees the objects whose information is included in the navigational instruction.
It is possible to calculate the position of a user relative to the positions of landmarks. In landmark-based positioning, positions of pedestrians are sensed or measured with respect to a landmark (Basiri et al., Reference Basiri, Amirian and Winstanley2014a). Relative position of a user with respect to a reference/fixed point can be measured using ultrasound (Holm, Reference Holm2009), dead-reckoning (Etienne and Séguinot, Reference Etienne and Séguinot1993), collaborative positioning techniques (Lee et al., Reference Lee, Grejner-Brzezinska and Toth2012) or inertial sensors (Vepa, Reference Vepa2011). The reference point in this project is a landmark, whose location is mostly fixed.
It is possible to measure/sense relative position of a user with respect to a landmark using the above-mentioned techniques. This paper proposes using feature-matching techniques to measure relative location of a user with respect to the landmark. Then, by using the absolute position of the landmark, stored in a database, it is possible to find the absolute position of the user. Since landmarks can be detected and labelled both indoors and outdoors (see Figure 1), it is possible to determine positions of users indoors and outdoors (Millonig and Schechtner, Reference Millonig and Schechtner2005). This paper shows the use of image processing and feature matching techniques in landmark-based position. Users can take a photograph of a registered landmark and send it to a localisation engine for image processing and feature matching. Then, based on feature extractions and feature matching techniques, it is possible to detect which landmark was photographed by the user. Then using scale and rotation (angle of view) of the photograph, it is possible to calculate the relative location of the user with respect to the detected landmark. As the absolute position of the landmark had been measured precisely and stored in a database, the absolute position of the user can be calculated and sent to the user, or used as an input variable for other services such as path finding and navigation.
Indoor landmarks.
Having the position of the user, the routing service can calculate the best path. This paper assumes that users look for the shortest path, which passes the maximum number of landmarks. This may provide users with a more reliable and interesting path as users can easily follow the instructions and also means that they will see more landmarks on the way.
In addition, it is possible to provide users with a photograph of to-be-seen landmarks as a part of their navigational instructions. This way of navigating is called landmark-based navigation (May et al., Reference May, Ross and Bayer2005). Landmark-based navigation is a kind of navigation service in which users are provided with navigational instructions, such as turn right, go straight, turn left, etc. whenever they approach each landmark (Fang et al., Reference Fang, Li, Zhang and Shaw2011). This is more compatible with human-to-human interactions and the interests of tourists and visitors (Basiri et al., Reference Basiri, Winstanley and Amirian2013).
In order to implement the proposed landmark-based navigation service, the National University of Maynooth's (NUIM) campus was considered. Buildings and important features, such as statues, historic monuments and buildings with unique architecture were stored in a spatial database as outdoor landmarks. Indoor landmarks such as main entrances and unique decorative objects and boards were also stored. For each landmark at least four photographs from different points of view but with the same distance and tilt were captured. Attributes such as names in English, names in Irish (Gaelic), age, feature category and some descriptive information were also collected and stored in the database.
Users' locations can be found using one of following positioning techniques: Global Positioning System (GPS), QR Codes or feature matching between photographs of landmarks taken by users and those in the database. Usually GPS is applicable outdoors and image-based localisation and QR Code techniques are used for indoor positioning. This paper explains how image-matching techniques can be used for indoor positioning. Users can take a photograph of a registered and labelled landmark and send it to a web service provider for further processing. Using image processing, feature extraction and feature matching, and thanks to the uniqueness of the landmarks, it is possible to find the landmark corresponding to the one in the database. Then using scale and rotation of the photograph taken by the user (which can be measured by the landmark's size and tilt), the relative position of the user with respect to the landmark can be calculated.
After finding a user's absolute location, the best path can be calculated from that point to the specified destination. This paper takes a combination of parameters, including minimum distance and maximum number of landmarks, to calculate the best route between two points. Then the user is given the navigational instructions, which contain text and pictures of the nearest landmark to be seen on the way. In showing a picture of a landmark, location and heading (direction) of user movement are considered to find the most similar picture to the view of the user among available pictures (Cao and McDonald, Reference Cao and McDonald2012). Receiving the picture of the nearest landmark on the way, users can also enjoy a guided tour by getting information about each building while navigating.
This paper explains how landmarks can be used for positioning purposes and how they can be provided as a part of navigational instructions. The second section is dedicated to landmark definitions and attributes. It explains what the important features and properties of landmarks are; this will help to identify landmarks with a more consistent approach. Positioning, path finding algorithms and navigational instructions based on landmarks are also explained in this section. Section three focuses on implementation of a pedestrian navigation application using landmarks.
2. USE OF LANDMARKS IN PEDESTRIAN NAVIGATION
A pedestrian can have several possible navigational strategies to find a desired goal (Redish, Reference Redish1999):
• The individual has no information and is forced to search randomly (random navigation);
• The individual moves towards a visible cue which leads to the arrival point (taxon navigation);
• The individual follows a fixed motor program (praxic navigation, e.g. “turn left after 200 metres, then turn right after 150 metres”);
• The individual associates directions with visual cues (route navigation, e.g. “turn left at the church”);
• The individual forms a mental representation of the surroundings and is able to plan routes between any locations within the area (locale navigation).
Landmarks have an important role in random navigation, taxon navigation, route navigation and locale navigation. Several researchers in the field of spatial cognition assert that navigating humans rely on three forms of spatial knowledge: landmark, route and survey knowledge (Siegel and White, Reference Siegel, White and Reese1975; Werner et al., Reference Werner, Krieg-Brückner, Mallot, Schweizer, Freksa, Jarke, Pasedach and Pohl1997). Exploring an unfamiliar environment, pedestrians first notice outstanding objects or structures at fixed locations. These unique objects and/or features are easy to recognise and can be kept in memory without difficulty (Schechtner, Reference Schechtner2005) so they are essential in route navigation and locale navigation strategies. Landmarks become more recognisable and therefore helpful for navigation when people move slowly, which is the case for pedestrians. The importance of landmarks for pedestrian navigation and wayfinding instructions is proved by many studies (Michon and Denis, Reference Michon and Denis2001; Denis, Reference Denis, Kuhn, Worboys and Timpf2003; Lovelace et al., Reference Lovelace, Hegarty, Montello, Freksa and Mark1999; Raubal and Winter, Reference Raubal, Winter, Egenhofer and Mark2002).
Landmarks are stationary, distinct and salient objects or features, which serve as cues for structuring a mental representation of the surrounding area. Any object can be perceived as a landmark if it is unique enough in comparison to the adjacent items. In this section landmarks and landmark-based navigation concepts and related definitions are explained. Firstly landmark definition and attributes are discussed. Then use of landmarks in positioning, path finding and providing navigational instructions are explained.
2.1. What is a Landmark?
A landmark can be defined as any object or feature, which is easily recognisable, such as a monument or a building. Landmarks are one of the interests of tourists, most often due to notable physical features, cultural references or historical significance. People often use landmarks for casual navigation, such as when giving directions verbally or when sketching a route map.
In urban studies as well as in geography, a landmark is furthermore defined as an external point of reference that helps orientation in a familiar or unfamiliar environment (Lynch, Reference Lynch1960). As noted above, landmarks are also used in verbal route instructions. These two properties of landmarks (i.e. being used as references to orient objects and being used in verbal route instructions) are very important and potentially helpful in providing more desirable navigation services.
Landmarks can be geometric shapes, and they may include additional information (e.g., in the form of bar-codes, QR Codes and geo-tags). In general, landmarks have a fixed and known position, relative to which users can localise themselves. Landmarks should be carefully chosen to be easy to identify; for example, a large building has priority over a small one. A feature, which has significant contrast to the background, is a good option to be considered as a landmark since its image would be easily recognisable to users. Such objects have to possess a certain saliency, which makes them remarkable and distinctive. So the surrounding area determines the characteristics a point must have to be perceived as a landmark (e.g. a shopping centre may not be very outstanding in urban areas, but becomes a salient landmark when being situated in a rural village).
After extracting and storing important features as landmarks, it is possible to provide an image-matching service for positioning purposes. The next subsection is focused on landmark-based localisation.
2.2. Use of Landmarks in Image-Based Positioning
Positioning is one of the most important components of any navigation system. Pedestrians need a seamless indoor and outdoor positioning service, so the system needs reliable and accurate positioning techniques in the situation where signals from Global Navigation Satellite Systems (GNSS) are not available. Unlike cars, pedestrians can easily enter buildings; so an ideal navigation system should work seamlessly indoors and outdoors. This paper discusses a landmark-based positioning technique, which can be implemented as a seamless indoor/outdoor positioning solution, since landmarks are available indoors and outdoors.
In general, two main categories of positioning techniques, which use landmarks as reference points, can be imagined; image-based and non-image-based techniques. Some examples of image-based landmark positioning are Quick Response (QR) Code-based positioning and photograph-based positioning. Examples of non-image-based positioning are Radio Frequency Identification (RFID) and Bluetooth network positioning.
This paper focuses on image-based landmark positioning techniques, since most mobile phones are now equipped with cameras so hardware requirements are fewer. RFID tags and Bluetooth networks are not available ubiquitously, therefore hardware needs to be embedded and installed both on users' handheld devices (e.g. RFID readers) and also on the landmarks (e.g. RFID tags). This means extra costs for both service provider and users. In addition, in most image-based positioning approaches the computation and processing phase is done on the server side so such approaches have lower power consumption in comparison with Bluetooth positioning and similar techniques where users need to keep their mobile phones' Bluetooth on all the time. In addition, many landmarks are registered as tourist and/or historic features. Consequently, it is almost impossible to install or affix a tag or any signal transmitter for positioning purposes. Landmarks can be huge objects; therefore a single tag or transmitter cannot represent its accurate location. Being much cheaper, having less power consumption and possessing the ability to be used on readily available mobile devices may make image-based positioning more accepted by users.
This paper uses an image-based technique for positioning which applies a feature-matching engine to identify landmarks of which users have taken photographs.
Users can take a photograph of a registered and previously stored landmark and then send/upload it for image processing and feature matching. The feature-matching engine can extract and identify the landmark of which the photograph has been taken. To do so, image processing, feature extraction and feature matching techniques are applied. In addition, the uniqueness of landmarks hugely improves the accuracy of feature matching results. Also, for each registered landmark, there is a QR Code generated and affixed nearby or on the landmark that can be scanned (or even photographed) by the user; in the case that the feature matching service cannot accurately detect the landmark due to existence of a similar object in another area or high number of requests, the QR Code can limit the area of search for the feature-matching engine (Basiri et al., Reference Basiri, Amirian and Winstanley2014b). This backup method helps the engine to provide better performance. Then scale and rotation (tilt) of the photograph can be calculated, since the absolute sizes of different façades of the landmark have already been stored in a database or can be measured from the images in the database. Based on scale and rotation, the vector representing relative location of the user with respect to the landmark can be calculated.
Positioning of a user based on a camera positioning system generally has two main steps; image processing to identify the landmark and finding scale and rotation. The photograph taken by the user is processed to find a landmark. In this step the uniqueness of landmarks makes it much easier to find matching images in the database. Then, using the actual size and shape of the corresponding landmark, which has been stored in a database, it is possible to calculate the scale and rotation of the photograph taken by the user. This piece of information is used to calculate the user's relative position with respect to the landmark. Since absolute positions of landmarks are available in the database, the absolute position of the user can easily be calculated and then used in path finding and navigation services.
2.3. Use of Landmarks in Path-finding Algorithm
In order to find the shortest path between two points, path finding algorithms look for the minimum distance traversed. To find the fastest path, the route with minimum time is preferable. In pedestrian navigation the shortest path and the fastest path algorithms usually have similar outputs, as there is generally no traffic factor for pedestrians and they move with almost unchanging speed. It is, however, possible to consider other parameters for path-finding and so get different results. For example, in the case of users not being familiar with the environment, they may look for the most reliable path rather than the shortest one. Taking the most reliable path may reduce the chance of getting lost. In addition, most tourist navigation application users want to see specific monuments and landmarks, which may need deviation from the shortest path. A landmark-based path-finding algorithm provides a more attractive and, at the same time, more reliable path. Since users see more landmarks on the way, they get less bored and they are also more certain that they are taking the correct route.
Landmark-based path-finding identifies the path which traverses the least distance to get to the destination while passing the most landmarks on the way. So landmark-based path-finding algorithms are trying to maximise the result of:
$$Number \; of \; landmarks \; of \; each \; edge\; / length \; of \; that \; edge.$$The same shortest path algorithm can then be implemented but the cost or distance, which is usually called weight and supposed to be optimised, will be replaced by the value of Equation (1) (i.e. number of landmarks/length of each edge).
Based on the landmark-based path-finding algorithm, once the route is calculated it is then possible to provide users with navigational instructions, which includes pictures of landmarks to be seen on the way.
2.4. Use of Landmarks in Navigational Instructions
As mentioned previously, landmarks can help to make navigational instructions approximate more closely the instructions that people give in their daily conversations. One of the best ways to use landmarks in navigational instruction is to provide images of landmarks as a part of the instructions at or near key points, such as turning points and important cultural or heritage features (landmarks).
For each key point in the calculated path (junctions or landmark to be seen), the direction (d) between the current key point or user's position (indexed i) and the following one (indexed i + 1) is calculated using Equation (2).
$$d = \pi /2 - \arctan _2 (lat_{i + 1} - lat_{i}\comma lon_{i + 1} - lon_i )$$Then, the image with the closest camera position from the key point i and the closest heading from d is retrieved. The landmark database is queried to retrieve a sequence of landmarks (position of camera when images were taken) from the desired path in the direction the user should take to reach the destination. Then based on these points, images are retrieved from the file system and then images are augmented by overlaying a 3D arrow pointing at the following direction to take, helping the user to visually interpret his or her position and get their bearings. The service uses two services behind the scenes:
• Simple image retrieval: This service takes geographical coordinates as an input and returns the URL of the image with closest registered coordinates within the database.
• Image sequence retrieval with visual navigation information: This service takes two GPS coordinates as an input, calls the path-finding service to get the best route between the two given positions, and returns the sequence of images from the database taken along this path using the Simple image retrieval service. Finally, the images are augmented with a 3D arrow pointing at the next direction in the path.
The implementation details of augmented reality functionality are described in the next section.
For each point of the calculated path, held in KML format, the direction d between the user's current position (indexed i) and the following key point (indexed i + 1) is calculated using Equation (2). Then, the image with the closest camera position from the point i and the closest heading from d is retrieved. To find the mentioned image, a score value s is calculated using location and heading from the Euclidean distance, which can be between 0 and 1 (Amirian et al., Reference Amirian, Basiri, Gales, Winstanley and McDonald2015). A sigmoid function is used to set to a score close to 0 for near images and 1 for the others. A trigonometric function is used to get a similar score for the heading (Equation (3)):
$$s = (1 - s_{pos} )(1 - s_{head} )$$In the above formula, the s pos is the score given by the Euclidean distance p between the position of the point and the position of image (Equation (4)):
$$s_{pos} = 1/(1 + e^{( - 2.25(\,p - 15))/5} )$$The constants of the Sigmoid function are set experimentally, representing the threshold distance from which it is possible to exclude the results as “they are too far away and cannot be a match”. Figure 2 shows a plot of this function. The higher the distance p is, the higher the cost is.
Position cost function where p is the Euclidean distance between the point positions (in the path) and the image camera position.
The heading score s head is calculated by Equation (5):
$$s_{head} = \vert\sin (h_{cam} + d)\vert$$Figure 3 shows a plot of this function. The higher the angle difference between the camera heading hcam and the heading direction d, the higher the cost is.
The heading score is high when the angle difference between the image camera heading and the heading direction d is high.
As is shown in this section, landmarks can be used for localisation, path-finding and navigational instruction.
3. IMPLEMENTATION
The landmark-based navigation system is designed to provide navigation services to pedestrians in the campus of the National University of Ireland. There are four main steps in our landmark-based navigation system design. Firstly, landmarks must be defined, extracted and stored in a spatial database. In this step both geometrical and non-geometrical characteristics of each landmark are stored in the spatial database. Another component of this system is the positioning component (Figure 5). The positioning component is responsible for calculating users' positions using image-based positioning based on photographs taken by users' cameras where GPS signals are not available; if GPS signals are available, then GPS gives the location. In both situations, availability and unavailability of GPS signals, it is possible to use the image-based positioning service; however it is not recommended to use it where GPS signals are received. This is done to minimise the number of requests users make, firstly to reduce users' devices' battery consumption, secondly to reduce network data exchange and thirdly to prevent any potential problem affecting the positioning component.
Then, based on a routing algorithm, which has the start point and the destination point as two inputs, the best path is calculated. Finally, navigational instructions, which help users to get to their destination, are provided using landmark information and photographs along the way. This section explains these four steps and their implementation into the campus navigation application.
3.1. Landmark Extraction
Landmarks have to possess a certain saliency, which makes them remarkable and distinctive. To decide which features can be stored as landmarks the project team members, as experts in urban planning, identified and short-listed the landmarks on the campus. Selected features, as landmarks, are stored in a spatial database as spatial objects with some attributes such as importance, type, name, etc. Landmark extraction and storage can be done automatically using the different criteria, algorithms and approaches discussed before.
In order to find and extract important features on the campus to be stored in the database as landmarks, three criteria were considered: uniqueness; contrast with surroundings; and size. Figure 4 shows one of the most important landmarks on the NUIM campus, extracted from a 3D model of the campus, which is the highest and the most unique feature and as such can be seen from almost everywhere on the campus.
Landmark extraction from 3D model of NUIM campus.
In order to select landmarks, one of the best approaches is comparing the attribute values of data records (e.g. size, colour, types, building labels, etc.) and identifying the most suitable landmark by ranking all attributes at potential decision points, as proposed by Elias (Reference Elias2003). The only problem in this approach might be that it disregards the fact that the visibility of an object is not invariable and that the quality of a landmark can differ depending on the direction from which it is viewed (Brenner and Elias, Reference Brenner and Elias2003). In order to address this issue, the visibility of each feature should be calculated using a Digital Surface Model (DSM) of the campus, which can be found from the 3D Campus project (Figure 4). Features, which can be viewed from long distances, are considered as suitable landmarks based on which we can provide landmark-based navigational instructions.
Knowing the user's height and the height of each landmark to be seen, it is possible to calculate whether or not the landmark is visible from the user's point of view.
Seeing a bigger part of a landmark increases the certainty and confirms that the user is on the correct path. Equation (6) shows the visibility of an object. If there is no intervening object that acts as an obstacle, then you can see the landmark 100% and if the intervening object is as high as the landmark then (L 2 − H)/L 2 is zero and you cannot see it. Thus (L 2 − H)/L 2 is a factor that shows the visibility of a landmark.
$$(L_2 - H)/L_2 *((D_1 *L_2 + D_2 *L_1 )/(D_1 + D_2 ) + 1)$$where L 1 is user's height and L 2 is the height of the landmark to be seen. H shows the height of the intervening object and D 1 shows the distance between the user and the intervening object. D 2 is the distance between the intervening object and the landmark to be seen.
Assuming L 1 to be approximately 160 centimetres, the average human height, it is possible to calculate the visibility of each landmark from users' point of view. This is used to provide information on landmarks, which can be seen from the users' point of view, as a part of the navigational services.
For each landmark, the edge(s), size and shape are also stored. These attribute values are applied by the landmark-based positioning component to calculate scale and rotation of the photograph taken by the user.
3.1. Landmark-based Navigation
Using landmarks stored in the spatial database, it is possible to find the best path and navigate users to their destination. The architecture used to provide the user with landmark-based navigational instructions is illustrated in Figure 5. The landmark-based navigation system implemented in the National University of Ireland consists of four main components; positioning component, data and service database, navigation service calculation engine and users and their mobile devices.
High-level conceptual architecture of eCampus applications.
The positioning component is responsible for calculating the position of the user using GPS or image-based positioning techniques, which were explained previously, and also for tracking them. It delivers its output, position of the user, to the navigation service calculation engine.
The navigation service calculation engine uses the user's position as an input for two other services; firstly, it calculates the best path using the user's position and selected destination, and secondly, based on the user's position, it can calculate visible landmarks on the way. In order to do both of these tasks, the navigation service calculation engine needs to have access to the spatial database where landmarks' information, such as location and size, is stored. In this project, a graphical database is used.
In general, elements in a landmark-based navigation service change with quite a high degree of frequency, since landmarks can be visible from one user's position but not from another's. Processing of such highly connected and interrelated data can be problematic for most conventional and modern storage systems such as Relational Database Management Systems (RDBMS). By nature, navigation services need to be highly responsive in real time or near real time. In addition, since there are usually a large number of users, such services must provide appropriate scalability and performance measures (Amirian et al., Reference Amirian, Basiri and Winstanley2014). Also, some of the network processing technologies process widely used network analysis and store these results beside the network data itself. However, when additional network elements (such as new landmarks or new roads) have to be added to the data, the previously built network must be rebuilt and recompiled. This recompilation process would be a serious processing task, especially when the newly added elements have lots of connections to the existing elements. In order to avoid such issues, it is better to store connected data in their natural representation and this is where the graphic database can come into play.
The navigation service calculation engine uses landmarks, edges and nodes data to find a more reliable path. This is calculated based on the landmark-based path finding algorithm explained in previous sections. Figure 6 shows one of the calculated routes; as is shown, some parts of calculated route pass through buildings. In order to calculate the user's position inside the building, image-based positioning is used since there are some stored indoor landmarks as well.
Navigation services in eCampus web application.
Once the route has been calculated, the user should be provided with navigational instructions to follow the calculated route. In this step, the navigation service calculation engine uses positional information provided by the positioning engine and also data stored in the database such as information on landmarks to calculate the landmarks to be seen from the user's location. Whenever the user's location changes or they request a new image-based positioning solution, this process will be repeated and a new set of navigational instructions is provided.
In Figure 6, the map on the user's screen is shown. As it illustrates, some of the landmarks are located outdoors and the rest are located inside. Upon reaching each landmark, related information is delivered to the user.
Since the navigational instructions contain some information about landmarks, which can be seen from the user's point of view, this approach may be more compatible with tourist navigation applications. In addition, users feel that it is less likely that they may get lost, since they are being provided with pictures of landmarks, to be seen shortly. So they are reassured that they are on the right path.
3.2. Providing Navigational Instructions using Landmarks
In order to provide image-based navigation instructions, a web service was developed which takes two coordinates as an input; current location (or origin) and destination. It then calls the path-finding web service to obtain the pedestrian path between the two given positions, origin and destination. Next it queries the path-finding service, and parses the KML response to find landmarks to-be-seen on the way. Figure 7 shows screen shots of the user's device showing visual navigational instructions.
Visual navigational instructions in eCampus web application.
The landmark-based navigation service takes advantage of the geolocation capabilities of mobile devices and/or HTML5 geolocation API and uses the pedestrian navigation web service in order to provide visual navigation on the NUIM campus. The position of the user is retrieved from the GPS of the mobile device or a QR Code affixed to each turning point or geolocation API (or manually selected by the user) and a desired destination is selected. The path between these two positions is retrieved using the pedestrian navigation web service. Then a spatial database of images of the campus is queried to retrieve a sequence of points (position of camera when images were taken) taken from the desired path in the direction the user should take to reach his destination. Then, based on the points, images are retrieved from the database and augmented by overlaying a 3D arrow pointing at the next direction to take, helping the user to visually interpret his position and get his bearings. Two services are used behind the scenes:
• Simple image retrieval: This service takes geographical coordinates as an input and returns the URL of the image with closest registered coordinates within the database.
• Image sequence retrieval with visual navigation information: This service takes two GPS coordinates as an input, calls the pedestrian navigation web service to get the pedestrian path between the two given positions, and returns the sequence of images from the database taken along this path using the Simple image retrieval service. Finally, the images are augmented with a 3D arrow pointing at the next direction in the path.
The whole process of positioning and retrieval is 800 ms, on average. The spatial database is implemented using PostgreSQL/PostGIS. Table 1 illustrates the schema of data in the spatial database.
Schema of image data table.
In order to capture images, a native mobile app is implemented which captures images along with the coordinates and heading of the mobile device. Then another script is used to upload data from the mobile device to the spatial database on the web server, as well as to transfer the images to a public directory on the web server.
Simple image retrieval is implemented using php-scripting language. To use this service a GET request can be sent to its URL with the latitude and longitude parameters (a geographical position). It returns a JSON string with the URL of the retrieve image in the public directory.
If no image can be found, it returns a 204 HTTP status code which means “No Content” is available.
The web application of image-based NUIM campus navigation services is available on http://147.252.87.12/eCampTest/Routing/ and the web application of image retrieval is available on http://ggalessrv.cs.nuim.ie/eCampus/public_scripts/imnav.html.
4. CONCLUSION
Landmarks are often used for casual navigation, such as giving verbal directions, by ordinary people. This can be implemented in navigation applications as well to provide users with landmark-based navigational instruction. Landmark-based navigation is a kind of navigation service in which positions of users are calculated based on the nearest landmark. Based on the current position of the user, which can be calculated using an image of the nearest landmark and the selected destination, a landmark-based path finding algorithm, which calculates a route with maximum landmarks on the way and minimum distance passed, calculates the most reliable path. Then users are provided with some information about interesting or important features, landmarks, that are nearby. This also provides reassurance that they are on the correct path. This paper explains these three steps in detail and demonstrates that they can be implemented successfully in a landmark-based navigation application.
ACKNOWLEDGEMENT
The research presented in this paper was funded by a Strategic Research Cluster grant (07/SRC/I1168) by Science Foundation Ireland under the National Development Plan. The authors gratefully acknowledge this support. This work was financially supported by EU FP7 Marie Curie Initial Training Network MULTI-POS (Multi-technology Positioning Professionals) under Grant Number 316528.
