Abstract: In urban AR, the automatic acquisition of scene structure is significant to adaptive information representation, and is a key step to solve the problems of unclear information indication and confusing scene perception caused by “cascade” information representation. However, the scene structure information is hidden in the scene image, which is difficult to extract directly. The 2D map, containing the locations, contours and spatial layout of the geographic entities, can be used as the prior information of the scene structure extraction. Aiming at the problem of difficult and inefficient acquisition of scene structure in urban AR, a method of automatic extraction for building scene from image using 2D map is proposed. Firstly, on the basis of geographic registration, some structural clues for scene image semantic segmentation are constructed, according to the contours and attribute information of buildings in 2D map. Then they are used as the parameters of image merging after over-segmentation to generate multiple image regions containing semantic information. Finally, according to the mapping relationship between the image regions and the building contours in the 2D map, the scene structure information such as the region contours, scene depth and plane orientation in the scene image is extracted. 32 building scenes in Graz area were selected for testing. The results show that the proposed method can extract scene structure in real-time with high accuracy, and the quality of the building facades extraction is obviously better than that of the comparison methods. The extracted scene structure can be applied to various adaptive information expression methods in urban AR to make the information annotation and the occlusion relationship clearer.

Key words: urban augmented reality, building scene structure extraction, semantic segmentation, adaptive representation, information annotation