Johns Hopkins APL, IARPA Explore Trojan Defense Methods for AI Training Process

Johns Hopkins APL, IARPA Explore Trojan Defense Methods for AI Training Process
artificial intelligence

Johns Hopkins University Applied Physics Laboratory has partnered with the Intelligence Advanced Research Projects Activity to identify new approaches to defend the artificial intelligence training pipeline against malware.

APL researchers are working with the intelligence community for an IARPA project aimed at leveraging deep neural networks to prevent Trojan attacks during AI learning processes, the lab said Friday.

Under the TrojanAI effort, APL and IARPA developed algorithms and used various network architectures to defend AI systems against “training-time attacks” that occur due to “backdoor” threats such as Trojans.

The National Institute of Standards and Technology also utilized the team’s open-source Python toolset for deep-learning models and deployed it at scale for testing against various detection scenarios.

“The AI supply chain will probably always have holes,” said Kiran Karra, a research engineer for the Research and Exploratory Development Department at APL. 

”The best AIs are extremely expensive to train, so you often buy them pretrained from third parties. Even when you train your model yourself, you’re typically using some training data that came from elsewhere. These are two prime opportunities to introduce Trojans.”

The TrojanAI team published details of the project in a report titled “The TrojAI Software Framework: An Open Source Tool for Embedding Trojans into Deep Learning Models".

Check Also

Bill Beard SVP Vertex Aerospace

Bill Beard Promoted to Corporate Operations & Strategy SVP at Vertex Aerospace

Bill Beard, former interim senior vice president of corporate business operations and development at Vertex Aerospace, has been promoted to SVP of corporate operations and strategy.


Carahsoft, GitLab Partner to Offer Agencies DevSecOps Platform via AWS Marketplace

Carahsoft Technology has agreed to offer GitLab's suite of DevSecOps tools to government organizations through Amazon Web Services' cloud marketplace and helped launch a product and training package as part of the partnership


QTS Receives EPA Recognition for Energy Procurement Approach

QTS Realty Trust has won a Direct Project Engagement award from the Environmental Protection Agency for the data center services provider's practice of sourcing renewable electricity to power its operations.