Can Foundation Models Grasp Space? ‘Theory of Space’ Benchmark Reveals Limitations
Stanford AI Lab has released ‘Theory of Space’ (TOS), a benchmark designed to test whether Foundation Models can construct, revise, and exploit spatial beliefs through active exploration. Across six state-of-the-art models, the research revealed a critical ‘exploration bottleneck,’ a persistent ‘text-vision modality gap,’ and severe ‘belief inaccuracies.’
TOS is engineered to measure the depth of understanding Foundation Models have of the real world, particularly concerning spatial relationships. Models are tasked with exploring virtual environments and learning object placements and movements, thereby testing their spatial reasoning capabilities. This benchmark is crucial for evaluating AI’s ability to truly comprehend situations and make informed decisions based on that comprehension.
The findings suggest that while current Foundation Models excel at learning from text and image data, they still face significant challenges in understanding spatial concepts in the physical world and engaging in active behaviors based on such understanding.