Epistemology of Embodied Representations

Signs Without Ground

On the inseparability of motion primitives
from their enactive context.

The separation of act from context is not a feature of reality —
it is a cognitive operation.
scroll to explore
Premise

A primitive is not a thing.
It is a relation.

Motion primitives extracted from robot sensorimotor experience have been proposed as grounded alternatives to language model tokens. We accept the data-efficiency argument. We contest the grounding claim.

A Best Matching Unit weight vector records what the body did. It does not record why that bodily response was the right one — the surface properties, object fragility, task constraints, and spatial affordances that shaped it. The environment is gone. What remains is a sign.

"The cup affords grasping for this hand, at this size, with this grip strength. Remove the agent specification and the affordance disappears."
— Gibson, adapted
Property LLM Token BMU Weight Vector Enactive Coupling
Grounded in linguistic co-occurrence body kinematics agent–environment relation
Context encoded corpus average body state only full coupling
Drift under transfer expected, acknowledged real, unacknowledged none (non-transferable)
Meaning origin relational physical + relational enacted
Diagram I

The Separation Operation

Reality presents a continuous flow of agent–environment coupling. There are no natural joints at which a "primitive" begins or ends. Segmentation is imposed by a cognitive system that needs to reason about and communicate behaviour.

How context is stripped when a coupling becomes a symbol
ENVIRONMENT surface · mass · affordances · task constraints · spatial layout · time pressure · fragility AGENT BODY joint velocities · finger pressures · palm orientation · object-relative position ⟵ continuous coupling ⟶ imposed segment [ 210 → 420 ] W(r*,c*) ∈ ℝ^ds kinematic only — context gone discarded discarded "Pick" transmitted symbol encode →
Reality: a continuous agent–environment coupling. There are no natural joints. The flow is unbroken.
Diagram II

Silent Meaning-Drift

When a symbol is transmitted, the sender abstracts from their context and the receiver reconstructs in their own. Neither party notices. The symbol appears shared. The enacted meaning is not.

Hover each element to reveal the hidden substitution
CONTEXT A sender's world SENDER enacts action stable surface rigid object no time pressure ample clearance CONTEXT B receiver's world RECEIVER reconstructs meaning compliant surface fragile object time pressure constrained space W(r*,c*) same symbol encode decode enacted meaning divergence — invisible to both parties
Hover an element above to see the explanation.
Diagram III

The Grounding Spectrum

From pure linguistic symbol to enacted coupling — four positions on the spectrum of representational grounding. Click each node.

Grounding completeness across representation types
symbolic enacted
LLM Token
5%
BMU Weight Vector
38%
(Body+Env) Primitive
72%
Enactive Coupling
100%
Select a node above to see what each representation encodes and where meaning-drift enters.
Diagram IV

Toward Inseparable Representations

Three necessary conditions for representations that do not silently drift. These are not optional improvements — they define the boundary between kinematic encoding and genuine action representation.

Click each requirement to expand
REQUIREMENTS FOR GENUINELY INSEPARABLE PRIMITIVES R1 Joint Encoding current: body only + env state required R2 Indexicality current: collapsed + context embedding R3 Transmission current: symbol only + context bundle encode transmit
Conclusion

The symbol travels.
The context does not.

The separation of act from context is not a feature of reality. It is a cognitive operation — useful, necessary for communication, but introducing silent meaning-drift whenever the separated symbol meets a new world.

This drift is silent because neither party has access to the substitution that occurred. The sender abstracts from their context. The receiver reconstructs in their own. The symbol appears shared. The enacted meaning is not.

Truly grounded representations — representations that carry their meaning with them — must encode the agent–environment relation, not the agent's body state alone. Building such representations is the deeper challenge that the motion primitive programme, at its most ambitious, should take on.

"The epistemological thesis developed here — that the separation of act from context is a cognitive operation rather than a feature of reality — is an original contribution of the first author, claimed as such in all contexts in which this work is cited or extended."