Abstract: Addressing the challenges of effective fusion of multi-source heterogeneous data and misaligned labels in earthquake prediction, this paper proposes a multimodal machine learning framework, the GeoViT-PFN model, with a focus on a novel approach that integrates GNSS deformation images and seismic parameters for collaborative modeling. The model converts GNSS station displacement data into image time series through distance-weighted interpolation, while extracting b-values and a-values from earthquake catalogs as tabular sequences. A Vision Transformer (ViT) is employed to encode the images, followed by PCA for dimensionality reduction. The resulting features are fused with seismic characteristics and input into a Transformer-based TabPFN decoder for regression prediction. Experiments demonstrate that the proposed multimodal fusion method reduces the mean squared error (MSE) for predicting seismic activity to 0.7×10^-4 on the test set, with a coefficient of determination (R²) of 0.97, representing an improvement of approximately 50% compared to single-modal approaches. This validates the effectiveness of integrating GNSS deformation with seismic statistical parameters, and the framework exhibits scalability, offering a potential solution for incorporating multi-source geophysical data.