arXiv cs.AI· Mohammad Sadeq Abolhasani, Viswanath Ganapathy·· 12 小时前AI 评分24
Griffin 如何通过合成数据课程学习实现先验数据拟合关系模型
Path to Prior-Data Fitted Relational Models via Learned Latent Multi-table Representations
AI 导读
图关系基础模型 Griffin 在大规模合成关系数据库上预训练后，仅将编码器潜表示交给冻结的上下文预测器，在 16 个真实关系分类任务上取得最佳冻结上下文性能。以 schema 引导数据起步、再转向 schema 无关数据的纯合成课程，优于仅用真实数据预训练及所有无序单阶段混合语料；接在真实数据预训练之后可得到整体最佳编码器。结果首次表明 PFN 式合成预训练可迁移到基于消息传递的图关系基础模型。
正文 · 原文
Abstract:Prior-data fitted networks (PFNs) show that pretraining on large populations of synthetic tasks can enable generalization to unseen datasets with little or no downstream fine-tuning. This paradigm is effective for transformer-based tabular and relational foundation models such as RDB-PFN, but remains largely unexplored for graph-based relational models built on message passing. We pretrain the graph-based relational foundation model Griffin on large collections of synthetic relational databases and ask whether curriculum learning, which benefits transformer-based relational learners, also improves message-passing representations. Rather than fine-tune on downstream tasks, we use the pretrained encoder as a general-purpose relational representation learner and pass its latent representations to a frozen in-context predictor (\ticl{}); no component is trained on the target task. On 16 real-world relational classification tasks, evaluated over five seeds against raw-feature and untrained-encoder baselines, a purely synthetic curriculum that begins with schema-guided data and continues with schema-agnostic data gives the strongest frozen in-context performance\rev{, outperforming real-data-only pretraining and every unordered single-stage mixture of the corpora; when appended after real-data pretraining, it yields the best encoders overall}. These results provide initial evidence that PFN-style synthetic pretraining transfers to graph-based relational foundation models.
| Comments: | Presented at the Fifth Learning on Graphs Conference (LoG 2026), Boston, MA, USA, November 20-22, 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.23073 [cs.LG] |
| (or arXiv:2610.23073v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23073 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mohammad Sadeq Abolhasani [view email]
[v1]
Fri, 16 Oct 2026 07:43:20 UTC (21 KB)
来源：arXiv cs.AI · arxiv.org