Lance and InstructX: unifying multimodal modelling and visual editing under one architecture
One paper unifies multimodal modelling through multi-task synergy; another routes visual editing through multimodal language model guidance. Both are attempts to stop maintaining a separate model per modality pair.
The engineering motivation is unglamorous and decisive. A separate model per modality pair means a combinatorial maintenance burden, inconsistent behaviour across paths, and no shared learning between tasks that are obviously related.
Multi-task synergy is the claim that has to be tested rather than asserted. Training tasks together sometimes helps and sometimes produces a model that is mediocre at everything — the difference depends on whether the tasks share structure, and that is an empirical question per combination.
Routing image editing through a language model's guidance is the more immediately practical of the two, because it turns editing into an instruction-following problem where the instruction can be checked.
arXiv — Lance: unified multimodal modeling by multi-task synergy → · arXiv — InstructX: towards unified visual editing with MLLM guidance →