From Topic to Transition Structure: Unsupervised Concept Discovery at Corpus Scale via Predictive Associative Memory
#unsupervised learning #concept discovery #predictive associative memory #corpus scale #topic modeling #transition structure #text analysis
📌 Key Takeaways
- Researchers propose an unsupervised method for discovering concepts in large text corpora using predictive associative memory.
- The approach moves beyond static topic modeling to capture dynamic transition structures between concepts.
- It operates at corpus scale, enabling analysis of massive datasets without manual labeling.
- The method leverages associative memory networks to predict concept sequences and relationships.
- This technique could enhance understanding of narrative flow and conceptual evolution in texts.
📖 Full Retelling
arXiv:2603.18420v1 Announce Type: new
Abstract: Embedding models group text by semantic content, what text is about. We show that temporal co-occurrence within texts discovers a different kind of structure: recurrent transition-structure concepts or what text does. We train a 29.4M-parameter contrastive model on 373 million co-occurrence pairs from 9,766 Project Gutenberg texts (24.96 million passages), mapping pre-trained embeddings into an association space where passages with similar transit
🏷️ Themes
AI Research, Natural Language Processing
Entity Intersection Graph
No entity connections available yet for this article.
Original Source
arXiv:2603.18420v1 Announce Type: new
Abstract: Embedding models group text by semantic content, what text is about. We show that temporal co-occurrence within texts discovers a different kind of structure: recurrent transition-structure concepts or what text does. We train a 29.4M-parameter contrastive model on 373 million co-occurrence pairs from 9,766 Project Gutenberg texts (24.96 million passages), mapping pre-trained embeddings into an association space where passages with similar transit
Read full article at source