Researchers have built a way to feed text, knowledge graphs, and hypergraphs into the same transformer without warping any of them into a flat token list.
The paper, posted to arXiv on September 18, introduces "labeled incidence structures" (LIS), a way to describe every data point - a word in a sentence, a node in a knowledge graph, an item in a hypergraph - with the same three-part label: its content, the role it plays, and the relation instance it belongs to. Because every format reduces to that same label, a single standard transformer can process text, knowledge graphs, and hypergraphs without converting any of them into a flattened token sequence first. The authors also prove a narrower technical point: cheaper approximations that just add a "position" term and a "relation" term separately cannot capture how position and relation interact, only their isolated effects. They apply the idea to knowledge stores too, showing that tying an entry's identity to its storage slot makes a model sensitive to storage order, while letting it learn identities freely becomes harder to control as the store grows.
Most transformer setups quietly force structured data - graphs, tables, relational records - into sequences, and that flattening throws away exactly the role and relationship information those formats exist to encode. A native way to keep text, graphs, and hypergraphs in one architecture matters for anything that mixes them, like retrieval-augmented systems pairing documents with knowledge graphs, or models trying to reason over both prose and structured facts at once.
It's a theory-heavy paper with proofs rather than benchmark wins, so the real test is whether LIS holds up once someone trains a full-scale model on it instead of a controlled example family.