Q-GeoMem: Question-Guided Geometric Memory for Video Spatial Reasoning

Abstract

Video spatial reasoning requires accumulating viewpoint‑dependent evidence over time while retaining information useful to the question being asked. Existing spatial video‑language models improve geometric perception and long‑range context modeling, but often treat memory as a generic temporal cache, which can introduce redundant or irrelevant geometry and weaken long‑horizon reasoning. We propose Q‑GeoMem, a question‑guided geometric memory framework for video spatial reasoning. Q‑GeoMem injects camera‑conditioned geometry into visual tokens and maintains two complementary memories: a Fine‑Grained Context Bank for recent dense features and camera states, and a Semantic‑Geometric Evidence Bank for compact long‑range evidence. Each candidate frame is assigned a calibrated Q‑Former‑based question relevance score, while novelty and evidence utility are recomputed with respect to the active evidence bank. The resulting relevance–novelty utility is used for capacity‑based replacement and as an attention bias during memory reading. During reasoning, both memories are read before update and adaptively fused with the current frame representation. Experiments on VSI‑Bench and VSTI‑Bench show that Q‑GeoMem achieves state‑of‑the‑art performance among evaluated spatial reasoning models, and ablations verify the contribution of the proposed evidence scoring mechanism.

Memory‑Centric Formulation

We recast video spatial reasoning so that camera‑conditioned geometry, question relevance, and novelty jointly decide what evidence should be retained.

Unified Architecture

Camera‑guided geometry fusion, a Fine‑Grained Context Bank for recent dense evidence, and a Semantic‑Geometric Evidence Bank for compact long‑range evidence, with adaptive fusion of memory readouts.

Relevance × Novelty

Selecting memory entries by both question relevance and novelty yields a more effective memory mechanism than treating memory as a chronological cache.

Interactive

See the memory think

The same egocentric video, viewed through multiple lenses of Q‑GeoMem. Switch the question to watch per‑frame relevance reshape, then step through streaming to watch novelty drive eviction in the evidence bank.

01 Question‑Guided Relevance

Each video keeps its own 32-frame axis. Switch questions within a video to watch the relevance curve interpolate between object-specific evidence.

Auto‑cycle questions

Question relevance r_i — how useful frame i is for the selected question

Auto‑cycle questions

Question relevance r_i — how useful frame i is for the selected question

02 Novelty & Eviction in SGEB

For the Bed dimension question, stream the video frame‑by‑frame. Each candidate is scored by novelty ν = 1 − max sim against the active bank; the relevance–novelty utility w = r·ν drives capacity‑based replacement.

Play/Pause

Step

Reset

Incoming frame

Press play or step to start streaming.

Semantic‑Geometric Evidence Bank capacity K = 6 · 0 stored

Stored-frame utilities are recomputed against the current bank at every step.

Incoming candidate history

novelty ν_t utility w_t = r·ν right scale evicted entry

Approach

How Q‑GeoMem works

Camera‑guided geometry fusion injects spatial cues into frame tokens; two complementary banks then read evidence before writing, and adaptive fusion combines both readouts with the current frame.

Q-GeoMem architecture overview. — **Overview.** (a) Camera‑Guided Geometry Fusion grounds frame tokens with camera‑conditioned geometry. (b) Question‑Guided Geometric Memory maintains the Fine‑Grained Context Bank and the Semantic‑Geometric Evidence Bank, where Q‑Former‑based relevance `r_t` and bank‑relative novelty `ν_t` compose the evidence utility `w_t`.

CGGF

Camera‑Guided Geometry Fusion

The visual stream attends to camera‑conditioned geometry through a reliability gate, so that both banks operate on spatially grounded features f_h,t.

FGCB

Fine‑Grained Context Bank

A sliding window of recent dense features and camera states. Camera‑feature differences modulate the read attention, supporting local, view‑dependent details.

SGEB

Semantic‑Geometric Evidence Bank

A fixed‑capacity bank of compact entries. Calibrated relevance r and novelty ν form the utility w = r·ν, used as a read‑time attention bias and for eviction.

Experiments

State‑of‑the‑art spatial reasoning

Q‑GeoMem achieves the best average score among evaluated spatial reasoning models on VSI‑Bench and VSTI‑Bench, while the same memory design transfers to out‑of‑distribution spatial, 3D QA, and long‑form video benchmarks.

71.2

VSI‑Bench avg.

+10.3 over VLM‑3R

67.8

VSTI‑Bench avg.

+9.0 over VLM‑3R

84.0

Camera Mov. Dir.

best non‑human

+34.5

Appearance Order

vs. VLM‑3R

VSI‑Bench

Method	Avg.	Obj. Cnt.	Abs. Dist.	Obj. Size	Room Size	Rel. Dist.	Rel. Dir.	Route Plan	Appr. Order
Gemini‑1.5 Pro	45.4	56.2	30.9	64.1	43.6	51.3	46.3	36.0	34.6
InternVL3‑78B	48.5	71.2	53.7	44.4	39.5	55.9	39.5	28.9	54.5
VLM‑3R‑7B	60.9	70.2	49.4	69.2	67.1	65.4	80.5	45.4	40.1
VLM²‑7B	68.8	72.5	59.6	70.8	69.9	69.0	87.8	52.6	68.3
Q‑GeoMem (Ours)	71.2	74.8	59.9	76.5	75.3	71.1	87.2	50.0	74.6

VSTI‑Bench

Method	Avg.	Cam‑Obj Abs. Dist.	Cam. Displace.	Cam. Mov. Dir.	Obj‑Obj Rel. Pos.	Cam‑Obj Rel. Dist.
GPT‑4o	38.2	29.5	23.4	37.3	58.1	42.5
LLaVA‑NeXT‑Video‑72B	44.0	32.3	10.5	48.1	78.3	50.9
VLM‑3R‑7B	58.8	39.4	39.6	60.6	86.5	68.6
VLM²‑7B	65.3	43.1	44.1	76.8	87.7	74.9
Q‑GeoMem (Ours)	67.8	44.3	44.6	84.0	91.0	74.9

Out‑of‑Distribution Generalization

Adding SGEB consistently improves transfer on video spatial reasoning, situated 3D scene QA, and general long‑form video understanding benchmarks.

Model	SPBench‑MV	MMSI‑Video	SQA3D	VideoMME	EgoSchema
Q‑GeoMem (w/o SGEB)	69.03	23.97	49.43	59.77	49.81
Q‑GeoMem (Ours)	72.61	25.49	50.38	60.96	53.11
Gain	+3.58	+1.52	+0.95	+1.19	+3.30

Core Component Ablation on VSI‑Bench

Model	Avg.	Obj. Cnt.	Abs. Dist.	Obj. Size	Room Size	Rel. Dist.	Rel. Dir.	Route Plan	Appr. Order
LLaVA‑NeXT‑Video‑7B ft. (Baseline)	64.83	71.91	51.47	75.38	70.56	67.32	72.09	42.27	67.64
Baseline + CGGF	67.28	72.07	56.13	74.49	72.43	68.17	81.47	45.36	68.12
Baseline + CGGF + FGCB	67.92	71.70	55.97	75.47	73.75	66.48	84.62	46.91	68.45
Baseline + CGGF + FGCB + SGEB	71.17	74.76	59.89	76.55	75.28	71.13	87.18	50.00	74.60

Camera‑Delta Read Modulation

Model	VSI Avg.	VSI Short	VSI Mid	VSI Long	VSTI^† Avg.	VSTI^† Short	VSTI^† Mid	VSTI^† Long
Q‑GeoMem (w/o SGEB), w/o Camera‑Delta	66.57	68.91	67.66	64.50	51.17	51.08	51.39	51.60
Q‑GeoMem (w/o SGEB)	67.92	70.55	69.43	65.33	52.24	53.25	52.16	52.32
Gain	+1.35	+1.64	+1.77	+0.83	+1.07	+2.17	+0.77	+0.72

SGEB Policy and Capacity

Policy	Update Rule	Read Bias	VSI Avg.	VSI Long
FIFO	evict oldest	none	68.7	66.3
QRel‑only	evict min r	βr	69.3	68.2
Novelty‑only	evict min ν	βν	68.9	67.3
Q‑GeoMem	evict min rν	βrν	71.2	70.1
Gain vs. FIFO			+2.5	+3.8

K	VSI Avg.	VSI Long
4	69.0	67.1
8	71.2	70.1
16	68.4	66.9
24	69.5	68.3
32	67.9	66.2

Length-based memory diagnostics for Camera-Delta modulation and SGEB memory strategy. — **Length‑based memory diagnostics.** Camera‑Delta modulation improves FGCB readout on VSTI‑Bench^†, while SGEB outperforms FIFO memory on VSI‑Bench with larger gains on longer videos.

Question-conditioned SGEB evidence selection on the same video for bed and TV dimension questions. — **Question‑conditioned SGEB evidence selection.** On the same video, Q‑GeoMem retains different long‑range evidence for bed and TV questions, showing that SGEB stores question‑relevant and non‑redundant observations.

BibTeX

@article{gao2026qgeomem,
  title   = {Q-GeoMem: Question-Guided Geometric Memory for Video Spatial Reasoning},
  author  = {Gao, Xianqiang and Chen, Qizhi and Qu, Delin and Song, Haoming and Wang, Zhigang and Zhao, Bin and Wang, Dong and Li, Xuelong},
  journal = {arXiv preprint arXiv:2605.27318},
  year    = {2026}
}

Q‑GeoMem Question‑Guided Geometric Memory for Video Spatial Reasoning

Abstract

Memory‑Centric Formulation

Unified Architecture

Relevance × Novelty

See the memory think

01 Question‑Guided Relevance

02 Novelty & Eviction in SGEB

How Q‑GeoMem works

Camera‑Guided Geometry Fusion

Fine‑Grained Context Bank

Semantic‑Geometric Evidence Bank

State‑of‑the‑art spatial reasoning

VSI‑Bench

VSTI‑Bench

Out‑of‑Distribution Generalization

Core Component Ablation on VSI‑Bench

Camera‑Delta Read Modulation

SGEB Policy and Capacity

BibTeX