Johns Hopkins APL, IARPA Explore Trojan Defense Methods for AI Training Process

Johns Hopkins APL, IARPA Explore Trojan Defense Methods for AI Training Process
artificial intelligence

Johns Hopkins University Applied Physics Laboratory has partnered with the Intelligence Advanced Research Projects Activity to identify new approaches to defend the artificial intelligence training pipeline against malware.

APL researchers are working with the intelligence community for an IARPA project aimed at leveraging deep neural networks to prevent Trojan attacks during AI learning processes, the lab said Friday.

Under the TrojanAI effort, APL and IARPA developed algorithms and used various network architectures to defend AI systems against “training-time attacks” that occur due to “backdoor” threats such as Trojans.

The National Institute of Standards and Technology also utilized the team’s open-source Python toolset for deep-learning models and deployed it at scale for testing against various detection scenarios.

“The AI supply chain will probably always have holes,” said Kiran Karra, a research engineer for the Research and Exploratory Development Department at APL. 

”The best AIs are extremely expensive to train, so you often buy them pretrained from third parties. Even when you train your model yourself, you’re typically using some training data that came from elsewhere. These are two prime opportunities to introduce Trojans.”

The TrojanAI team published details of the project in a report titled “The TrojAI Software Framework: An Open Source Tool for Embedding Trojans into Deep Learning Models".

You may also be interested in...


Nimbix Unveils High-Performance Computing Offerings for Gov’t Customers

Nimbix has announced availability of its high-performance computing services to support cloud platforms compliant with the Federal Risk and Authorization Management Program, International Traffic in Arms Regulations and other government-imposed security standards. The company said Tuesday its JARVICE XE and HyperHub platforms offer HPC for artificial intelligence and simulation uses in defense, aerospace, energy and health care industries.

Tutor Perini

Tutor Perini Subsidiary Awarded $67M for Defense, Homeland Security Infrastructure Projects

A Tutor Perini subsidiary has booked $66.9 million in two awards for construction efforts with the departments of Defense and Homeland Security. Perini Management Services Inc. will renovate U.S. Air Force Academy's Cadet Field House in Colorado and design and build a dormitory at the Glynco campus for DHS.

Intelligent Artifacts

Intelligent Artifacts to Launch R&D on Sensor Fusion Tech for USAF

New York-based artificial intelligence company Intelligent Artifacts has secured a contract of an undisclosed sum through the first phase of the Small Business Innovation Research program to explore and develop a sensor fusion technology for the U.S. Air Force.