CoLVR: Enhancing Exploratory
Latent Visual Reasoning
via Contrastive Optimization
Give visual reasoning room to explore.
† Corresponding author
From fixed paths to possibilities.
Match predefined features.
A narrower space for reasoning.
Learn relative relationships.
Keep the freedom to explore.
over Mirage · 65.83% accuracy
over Mirage · 78.00% accuracy
over LVR (8 steps) · 61.47% accuracy
Less constraint.
More discovery.
Continuous hidden states.
More exploratory visual reasoning.
What if visual reasoning could explore beyond a predefined path?
Latent visual reasoning lets multimodal models think through continuous hidden states instead of decoding every intermediate step into text. Yet hard alignment to predefined visual features can compress these states into a homogeneous space, limiting exploration.
CoLVR—Contrastive Optimization for Latent Visual Reasoning—uses relative contrastive relationships to guide reasoning while preserving flexibility. Angle-based negatives shape the latent space during supervised fine-tuning, and a trajectory contrastive reward carries that principle into reinforcement learning.
Guide the direction of reasoning.
Leave room for the model to explore.
Read the full abstract
Due to the potential for exploratory reasoning of Latent Visual Reasoning, recent works tend to enable MLLMs (Multimodal Large Language Models) to perform visual reasoning by propagating continuous hidden states instead of decoding intermediate steps into discrete tokens. However, existing works typically rely on hard alignment objectives to force latent representations to match predefined visual features, thereby severely limiting the exploratory of latent reasoning process. To address this problem, we propose CoLVR (Contrastive Optimization for Latent Visual Reasoning). To obtain a more exploratory visual reasoning, CoLVR introduces a latent contrastive training framework. Firstly, CoLVR learns diverse and exploratory representations with a latent contrastive objective guided by angle-based perturbation, which expands the semantic latent space and avoids over-constrained embedding. Then, CoLVR employs a latent trajectory contrastive reward for RL (Reinforcement Learning) post-training to enable fine-grained optimization of latent visual reasoning process and thus fostering diverse reasoning behaviors. Experiments demonstrate that CoLVR significantly enhances the exploratory capability of latent representations, achieving average improvements of 5.83% on VSP and 8.00% on Jigsaw, while also outperforming existing latent models on out of domain benchmarks, with a 3.40% gain on MMStar. The data, codes, and models are released at github.com/Oscar-dzy/CoLVR.
Three stages.
One exploratory latent space.
Built on Qwen2.5-VL-7B.
Reasoning through 8 latent tokens.
Align latent hidden states with the mean-pooled input image representation. An auxiliary answer loss preserves language generation while the model acquires an initial visual reasoning capability.
Better reasoning.
Beyond the training task.
Explore the paper’s reported results.
Accuracy (%), higher is better.
Visual Spatial Planning
Overall accuracy across seen and unseen VSP levels.
| Model | VSP Total | Jigsaw | Tetris |
|---|---|---|---|
| Mirage | 60.00 | 70.00 | 43.33 |
| CoLVR (Ours) | 65.83 | 78.00 | 44.67 |
Source: main results table. All scores are accuracy (%). Improvements are absolute percentage points.
What does exploration look like?
Inspect the evidence behind the performance.
A more dispersed latent space.
CoLVR’s latent states are more spread out in the paper’s UMAP visualizations, while Mirage’s states form compact clusters. These 2D projections illustrate representation diversity; they are complemented by measurements in the original latent space.
Plan. Complete.
Imagine.
Three ways to put visual reasoning to the test.
01Visual Spatial PlanningFind a path through a visual maze
Navigate grid environments from start to goal. Levels 3–6 evaluate seen settings; harder levels 7–8 test generalization. Result-hint images provide guidance during training.
02JigsawRecover missing visual information
Identify the correct pieces for missing regions of an image. Solving the puzzle requires global visual context and the ability to imagine a completed scene.
03TetrisReason about constrained spatial completion
Select pieces that fill the target shape without gaps or overlap. This task tests spatial composition and structural reasoning. The source tables label this benchmark “Tertis”.
Continue the exploration.
Read the paper. Explore the code. Build on CoLVR.
Build on this work.
If CoLVR is useful for your research,
please consider citing our paper.
View on OpenReview ↗
@inproceedings{ding2026colvr,
title = {CoLVR: Enhancing Exploratory Latent Visual
Reasoning via Contrastive Optimization},
author = {Ziyang Ding and Linjian Meng and Yiming Wu and Yuhan Li and Yuhao Liu and Zhen Zhao},
booktitle = {The Fortieth Annual Conference on
Neural Information Processing Systems},
year = {2026},
url = {https://openreview.net/forum?id=FdggkQpSHq}
}