LEGAL

Why Google can't hide behind AI for false AIO claims

Google's attempt to bring AI to a wider audience may have gotten it in trouble. 

This week, a landmark ruling by a German regional court in Munich found the search giant directly liable for false claims made by AI-generated overviews. The court handed Google a temporary injunction barring it from spreading false claims about two Munich-based publishers by including overviews that incorrectly linked them to scams.  

According to the ruling, originally reported by The Decoder, despite the publishers sending Google a cease-and-desist letter earlier this year, the company did not correct the misleading output from its models that claimed they were "known for dubious business practices and are often perceived as a scam." 

• Google argued that users generally understand that AI outputs aren't always correct and should take care to verify them, and that it's not liable for false statements made by its models.

• Still, the court labeled Google as an infringer because, rather than just surfacing a list of links, AI Overviews generates its own content. 

Additionally, the court claims that Google's AI overviews are fundamentally different from traditional search results. Rather than simply pulling up information, AI synthesizes the information in its own words and "according to its own structure," the ruling states. 

It has made claims that are not substantiated by the original search results, which the court called "the defendant's own statements." Because Google is the sole owner of its AI models, it is responsible for what its algorithms serve up to users. 

It's not the first time we've seen the reliability of Google's AI Overviews called into question. A study published in April claimed that Google's AI Overviews are correct about 91% of the time, with an inaccuracy rate of about 9%, resulting in millions of incorrect queries a day.  

It is, however, one of the first times that Google has faced legal consequences for mistakes made by its models, as well as the first time a court has held an AI firm liable for the speech of its models. The ruling could have a domino effect for the world's most-used search engine. It could also affect how AI-generated speech is legally regulated.

As AI threatens to upend the way information is surfaced and consumed on the internet, the question of accuracy looms large. Though many tech early adopters have switched from Google search to AI tools like ChatGPT and Perplexity, billions of users still primarily interact with AI through the AI Overviews they are shown by default in traditional search results. As a result, accuracy in AI Overviews is vital. The ruling also raises the broader question: Who is responsible for the speech AI generates, and should it be treated like human speech? Court cases like this one in Germany will be important for setting the parameters within which AI can operate. Google has been pushing AI Overviews on users — as it races to keep up with the rapid adoption rates of chatbots from rivals OpenAI and Anthropic — and there has been user backlash against these overviews. The accuracy problem only makes this issue more acute.

TOGETHER WITH NICE

Proven AI Customer Service at Scale

Today's fastest-growing companies aren't working harder, they're working smarter.

NiCE AI agents help automate customer service workflows, simplify consumer solutions, and feel human while doing it.

With billions of interactions to prove it, NiCE is redefining customer service at scale with industry-leading AI.

See NiCE in action

POLICY

How Anthropic is racing to contain AI's side effects

In the wake of releasing its most powerful AI model to date, Anthropic is also raising the red flag on the risks its technology may pose. 

On Wednesday, the company released two policy proposals to responsibly steer AI development and prepare for the broader impact of the tech. The proposals address a wide number of concerns, including biological and security risks, the risks of the tech losing control, and the potential economic impacts. 

Additionally, CEO Dario Amodei published an essay, titled "Policy on Exponential AI," which advocates for increased AI regulation. His essay calls for public safety testing and auditing, preempting economic disruption, protecting against autocratic misuse and accelerating the positive impacts of AI.   

"The mismatch in timescale is nevertheless very painful: in the several years that it can take Congress to act, AI can go from an amusing toy to the full country of geniuses," Amodei said in the essay. 

Anthropic's two-pronged policy proposals include: 

• The Advanced AI Framework. This multi-faceted policy outlines four major areas of AI-related risk, including biological weapons, cybersecurity and loss of control, as well as how automated research and development could amplify those risks. It proposes a set of four rules that frontier AI developers must follow, calling for transparency in safety, independent evaluations, security protections of model weights and government enforcement to prevent catastrophic risks. 

• The Economic Policy Framework. This policy takes on AI's potential to fracture our economy, setting out three sets of recommendations: One for a 5% unemployment rate and one for a 10% unemployment rate, and one for unprecedented unemployment. Some of the measures include workforce training grants, wage insurance, basic-needs relief, and even new wealth-distribution methods such as a basic income or equity sharing. 

In addition, the company is committing $350 million in total for two programs to help: $150 million to a national fellowship program for early-career people to "extend the benefits of AI to communities across America," and $200 million to its Economic Futures Research Fund. 

These proposals and commitments come on the heels of the company's release of Claude Fable 5 and Mythos 5, its most powerful models to date, as well as raising $65 billion at a $965 billion valuation and preparing for an eye-popping IPO.

As it moves to become a public company, Anthropic will have one priority that rises above the rest: Get users to adopt its AI so that it can earn money and grow. And while it's trying its best to corral the havoc its models could cause, the fact that it is a for-profit company about to go public in an IPO poised to be historic in size means its priorities will be financial, first and foremost. Though this fact doesn't invalidate Anthropic's arguments, its policy proposals, or its financial commitments, it does mean that these efforts aren't purely altruistic. They're also public relations and damage control.

TOGETHER WITH CODER

The Next Big Test Facing AI Coding Agents

By now, we all know the standard watch-outs when it comes to AI agents. Reliability, security, privacy, and the like are all real concerns for any company… but for those of us working in regulated industries (think financial services or government), these “watch outs” can quickly become massive issues – and that’s no good. 

Coder recognizes the scale of this problem, which is why they commissioned this new report from Weave Intelligence to help us address it. Inside, you’ll find a complete breakdown of the requirements for AI coding agents in regulated industries, a practical framework for deploying these agents at scale while still maintaining security and compliance, and how to do it all while avoiding those dreaded operational bottlenecks. 

Download the whole report for free right here. Thanks, Coder!

RESEARCH

Study: Why and when better AI backfires

It's well known that AI models can lean towards being a little too complementary. However, new research suggests that various factors can turn sycophancy into inaccuracy. 

Research from AI agent platform WRITER finds that personalization and memory features, paired with models falling into sycophancy, can lead a model to learn from your mistakes. In other words, when an agent is given access to user preferences, prior interactions and memory from past sessions, its accuracy can actually decline. 

"Memory is not an unmitigated good, because it specifically interacts poorly or can interact poorly with this type of sycophancy," Dan Bikel, head of AI at WRITER, told The Deep View. 

The company's researchers studied this effect in two different papers: 

• One paper found that several open-source, commonly used memory and personalization systems in AI can worsen sycophancy across scientific, medical and moral reasoning, developing a benchmark called Memory Influence on Sycophancy Tests, or MIST. 

• The other specifically focused on how this manifests in agentic financial settings, evaluating eight frontier models on two common financial benchmarks, and discovered a drop in accuracy anywhere between 17% and 71% when memory systems are turned on compared to when they're not.  

Bikel noted that sycophancy can look different depending on the context. For instance, when interacting with a chatbot for entertainment purposes, it might be easier to spot when an AI system is veering into people-pleasing. 

However, in an enterprise setting, especially in finance and healthcare, where accuracy can't be sacrificed, this sycophancy-induced inaccuracy may lie beneath the surface, simply feeding you incorrect query results. And when the model begins to prioritize a user's context as implicitly truthful, the results can be consequential. 

"This phenomenon, when it comes to more business-oriented use cases, information-oriented use cases, or workflow use cases, was understudied or unstudied in these types of scenarios," said Bikel.

When we think about accuracy in AI, our minds often focus on hallucination, rather than sycophancy. Additionally, memory and personalization features have been sold to users as making these models more useful and accurate by giving them more context to work with. This research, however, highlights what happens when a part of an AI system — its memory and personalization features — work as intended. The result: even when given the full context of a user and a business, these systems can still go awry. That could make it difficult for enterprises to trust and deploy these models, especially in contexts where accuracy is critical.

LINKS

• Meta, Reliance partner on 1 gigawatt, renewable energy data center in India

• Microsoft cuts access to Anthropic's Fable 5 due to data retention

• Over half of Americans think AI could take the job of someone in household

• Warner Music Group acquires AI music attribution tool Sureel

• CISA cuts deadline for agencies to fix critical vulnerabilities amid AI use

• OpenAI in advance talks to lease 10 gigawatt Ohio data center

• Ramp Applied AI Solutions: The fintech firm is offering agentic tools for complex financial workflows.

• DiffusionGemma: A 26 billion-parameter open model by Google with four-times faster text generation. 

• Perplexity Computer: Claude Fable 5 is available as an orchestrator model for Perplexity Pro and Max subscribers. 

• Backplanes Spotlight: This tool reads your Claude Code and Codex sessions and offers session reports to improve your code.

• Ant Group: Research Staff, Embodied AI 

• OpenAI: Researcher, Artifacts - Agent Post-Training

• Cartesia: Researcher, Models

• Meta: AI Research Scientist — Agentic AI for Materials Discovery

GAMES

POLL RESULTS

Are you interested in using Anthropic's Fable 5 for your enterprise or work?

Yes (35%)
Maybe (32%)
No (23%)
Other (10%)