Deploying AI in any organization means granting it access to sensitive data and processes. Cisco wants companies to know exactly what they're letting in before they do.
On Thursday, Cisco launched a Model Provenance Kit, which it describes as a “DNA Test for AI models” that verifies a model's origins and checks whether it has been tampered with. The tool is ultimately supposed to give organizations more confidence about the models they deploy.
"Model provenance will underpin AI governance and AI security by making it possible to trace how systems are built, how they evolve, and how their outputs can be reliably attributed in high-stakes environments,” said Amy Chang, head of AI threat intelligence and security research at Cisco, told The Deep View.
Model Provenance Kit analyzes the model’s identity using architecture metadata, tokenizer structure, and learned weights to produce a “rich fingerprint” for each one, as well as a single provenance score indicating whether two models share a common origin or training lineage. There are two modes:
- Compare mode: Takes any two models and gives them a score that reflects how much they share in lineage.
- Scan mode: The user would begin with a single model and match it against a database of known fingerprints to get the closest lineage candidates.
The company evaluated its accuracy against a 111-pair benchmark, and only 4 out of the 111 pairs were misclassified, and those involved “extreme architectural transformations.” Model Provenance Kit is available today, and the repository is accessible via GitHub, and the model fingerprint database is available on Hugging Face.
This targets a major issue in the AI space: People are downloading open-source model repositories from platforms such as HuggingFace without being able to verify the model's exact origins. For instance, Cisco’s announcement highlights how a developer can claim a model was trained from scratch, yet it could still be a copy of another model.
Other possible issues include biases in the training data, vulnerabilities, licensing caveats, or models that have simply been modified during development without those changes being accurately logged. Cisco highlights that this makes companies vulnerable to poisoned or compromised models, licensing and regulatory risks, supply chain integrity risks, and incident response risks.
Our Deeper View
Cisco’s goal is to help enterprises adopt AI safely and securely by providing the tools to do so. A prime example is the recently released LLM Security Leaderboard, which ranks top models based on their responses to adversarial attacks and associated security risks. Its Model Provenance Kit is another. Together, these tools put Cisco in a strong strategic position: By helping clients understand the safety and integrity of the models they evaluate, Cisco becomes a natural partner for the infrastructure needed to deploy them.