Expressive Power of Graph Transformers via Logic
#graph transformers #GPS-networks #soft attention #hard attention #expressive power #real numbers #floating‑point #machine learning #graph neural networks #theoretical analysis
📌 Key Takeaways
- Study focuses on graph transformers (Dwivedi & Bresson, 2020) and GPS‑networks (Rampasek et al., 2022).
- Both soft‑attention and average hard‑attention variants are examined.
- Analysis is performed in two settings: (1) theoretical case with real numbers, (2) practical case with floating‑point values.
- The goal is to characterize the expressive capabilities of these models on graph data.
- Findings aim to inform both theoretical foundations and practical deployment of graph transformer architectures.
📖 Full Retelling
Researchers in machine learning have published a study on arXiv (arXiv:2508.01067v2) examining the expressive power of graph transformers (GTs) introduced by Dwivedi and Bresson (2020) and GPS‑networks described by Rampasek et al. (2022). The paper analyzes both soft‑attention and average hard‑attention mechanisms under two scenarios: a theoretical framework with real numbers and a more practical setting using floating‑point values. The aim is to deepen understanding of how much graph structure these transformer models can capture and to bridge the gap between theoretical insights and real‑world applicability.
🏷️ Themes
Graph Neural Networks, Transformer Architecture, Expressive Power Analysis, Attention Mechanisms, Theoretical vs Practical Evaluation
Entity Intersection Graph
No entity connections available yet for this article.
Original Source
arXiv:2508.01067v2 Announce Type: replace-cross
Abstract: Transformers are the basis of modern large language models, but relatively little is known about their precise expressive power on graphs. We study the expressive power of graph transformers (GTs) by Dwivedi and Bresson (2020) and GPS-networks by Ramp\'asek et al. (2022), both under soft-attention and average hard-attention. Our study covers two scenarios: the theoretical setting with real numbers and the more practical case with floats.
Read full article at source