GeoDecoder: Empowering Multimodal Map Understanding
CoRR(2024)
摘要
This paper presents GeoDecoder, a dedicated multimodal model designed for
processing geospatial information in maps. Built on the BeitGPT architecture,
GeoDecoder incorporates specialized expert modules for image and text
processing. On the image side, GeoDecoder utilizes GaoDe Amap as the underlying
base map, which inherently encompasses essential details about road and
building shapes, relative positions, and other attributes. Through the
utilization of rendering techniques, the model seamlessly integrates external
data and features such as symbol markers, drive trajectories, heatmaps, and
user-defined markers, eliminating the need for extra feature engineering. The
text module of GeoDecoder accepts various context texts and question prompts,
generating text outputs in the style of GPT. Furthermore, the GPT-based model
allows for the training and execution of multiple tasks within the same model
in an end-to-end manner. To enhance map cognition and enable GeoDecoder to
acquire knowledge about the distribution of geographic entities in Beijing, we
devised eight fundamental geospatial tasks and conducted pretraining of the
model using large-scale text-image samples. Subsequently, rapid fine-tuning was
performed on three downstream tasks, resulting in significant performance
improvements. The GeoDecoder model demonstrates a comprehensive understanding
of map elements and their associated operations, enabling efficient and
high-quality application of diverse geospatial tasks in different business
scenarios.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要