FAST: efficient action tokenization, and why the body needs a vocabulary
Vision-language-action models have to turn continuous joint commands into discrete tokens a transformer can predict. How you do that determines how much of the model's capacity is spent on the body rather than the task.
Naive discretisation is expensive. Bin every joint finely and the sequence length explodes; bin coarsely and the motion is jerky. Action tokenization is the compression problem sitting between a language model and a physical arm, and it has been solved badly for most of the field's history.
It is also where cross-embodiment transfer either works or does not. A tokenizer tied to a specific joint configuration produces a model that cannot move to another robot. One that captures motion primitives rather than raw angles has a chance.
The parallel to text tokenization is close enough to be useful. Byte-pair encoding was not the interesting part of language modelling and it determined a great deal about what those models could represent efficiently.
arXiv — FAST: efficient action tokenization for vision-language-action models → · arXiv — Hybrid training for vision-language-action models → · arXiv — SmolVLA: a vision-language-action model for affordable and efficient robotics →