Leveraging LLM for Movement Patterns-Aware Trajectory Representation Learning

Spatial-temporal trajectory mining plays a crucial role in location-based services. Trajectory representation learning transforms complex trajectory features into low-dimensional representation vectors, enabling various downstream tasks. This process requires effectively extracting spatial-temporal features and motion patterns from trajectories. However, trajectory representation learning faces numerous challenges due to limitations in model capacity and issues with the quality and scale of trajectory datasets. Large Language Model (LLM), pre-trained on massive high-quality datasets, have demonstrated remarkable capabilities in multi-task applications. Given the similarities between trajectories and natural language, it is feasible to develop an efficient trajectory representation method leveraging LLM. Nevertheless, vanilla LLMs cannot directly handle the unique spatial-temporal characteristics of trajectories or effectively extract motion patterns. To address these issues, we propose a trajectory representation learning model, MPTraj, which leverages LLMs to extract motion patterns. MPTraj introduces a novel trajectory semantic embedder that enables LLMs to process spatial-temporal features and extract movement patterns effectively. Additionally, a trajectory encoder with prompt tuning is integrated to generate high-quality trajectory representations. Furthermore, two pre-training tasks—next trajectory point prediction and movement pattern classification—are designed to enhance the model's performance on downstream tasks. Experimental results on two real-world datasets and four representative tasks demonstrate the effectiveness of MPTraj.