igger isn’t always better.
As large language models continue to add zeros to their parameter counts (and their costs), some developers may be setting their sights smaller. Small models are gaining traction as companies seek to cut costs and focus on niche AI applications.
While large language models have the benefit of general intelligence, a jack of all trades is a master of none. Small models, or those with parameter counts in the millions to low billions rather than the trillions, may outperform larger ones at niche tasks.
- These models have become a focus area for both Big Tech companies and startups, with Meta, Microsoft, Google and OpenAI all debuting their own mini offerings.
- Investors have taken a liking to the space, too: Companies like Mistral AI, Liquid AI and Sakana (which closed a $135 million Series B round on Monday) have all hit unicorn status as they rake in funding for lightweight and niche models.
And research from IBM released in recent weeks November, shows that these models may also be better at staying focused.
When comparing how small language models performed against their larger counterparts on a number of financial-related tasks, the small models were able to avoid drift, achieving “100% output consistency,” challenging the common belief “that larger models are universally superior for production deployment.”
The number one reason that these models are so popular is cost, Dan Herbatschek, CEO and founder of Ramsey Theory Group, told The Deep View. These models are both cheaper to build and run, often requiring fewer GPUs and the capability to operate with CPUs or mobile hardware. “This fits well with a small AI budget,” he said.
These models also tend to be more secure and less resource-intensive, making them easier to deploy in agents and edge cases where compute power is limited, Chuan Qiu, founder of Velda, told The Deep View. “Being real-time (and) protecting privacy are crucial for these cases,” Qiu noted.