he next step change in AI might not be large language models.

Spatial intelligence, or AI that can understand the world visually, might be the “next frontier” of the technology, Dr. Fei-Fei Li, a Stanford University computer science professor and founding co-director of Stanford’s Human-Centered AI Institute, wrote in an essay published Monday.

In the essay, Li, who founded large-scale visual learning benchmark ImageNet, called spatial intelligence “the scaffolding upon which our cognition is built,” She argues that that perception and action are the “core loop driving the evolution of intelligence,” and machines have yet to achieve the “fluency” of spatial intelligence, such as the ability to make split second decisions or communicate through gestures.

While significant strides have been made in the field, such as multimodal language models and video generation tools, “spatial capabilities remain far from human level.” The next frontier, she notes, are building world models, or those capable of understanding dynamically complex worlds, both real and virtual.

• A good world model has three qualities: physically consistent generation, understanding multimodal inputs, and the ability to output “next states.”
• While the scope of this challenge is wide, the gains could be massive. It could supercharge creativity in the short term, robotics in the mid term and education and scientific discovery in the long term.

“Our dreams of truly intelligent machines will not be complete without spatial intelligence,” she concludes.

Li’s essay highlights that the key to unlocking real value from AI might lie beyond just making language models bigger and better (despite the fact that some of the biggest model developers are keenly focused on besting each other's LLMs). A number of companies, both startups and Big Tech alike, have dedicated resources to boosting this intelligence.

And Li isn’t the first AI leader to make this declaration. Yann LeCun, chief scientist of Meta AI, made a similar point in an interview with journalist Alex Kantrowitz back in May, noting that “We are not going to get to human-level AI just by scaling up LLMs.”