[EXT] RE: Proposed new CWE: Machine learning classifier vulnerable to adversarial inputs (adversarial machine learning)
I agree that the hacking of AI is an emerging security crisis, and many will agree that hackers are on the brink of launching a wave of AI attacks.
Adversarial ML could certainly help to better specify criteria for new CWE IDs and CAPEC IDs. Specifying deterministic criteria for characterizing the associated weaknesses and attack patterns might be some of the most valuable contributions of this research community.
We could certainly specify new CAPEC IDs to characterize the spectrum of attack patterns depending on phases of ML model generation, such as training time attack or inference time attack. Knowing that ‘changes to the data from which ML systems are taught could also lead to biases being actively added to the decisions AI systems make’ could be specified as a CAPEC ID; yet detecting such actions (and characterizing the ‘source vector’ of attack) could be challenging.
The CWE/CAPEC research community could help the AI/ML research community by specifying new CWE IDs and CAPEC IDs that are broken out by white box access and black-box attacks. In order for the AI/ML community to devise appropriate defenses that eliminate or at least mitigate risks attributable to the existence of adversarial examples, they could greatly benefit from standardized specification of associated weaknesses and attack patterns.
As indicated in the referenced article: "When designing the machine learning systems, it is important to be aware of and possibly mitigate the specific risks of adversarial attacks, rather than blindly design the system and worry about repercussion if they happen."
New CWE IDs and CAPEC IDs associated with ML would certainly contribute to the advancement of safe and security AI/ML.